Image resolution and contrast enhancement with optimal brightness compensation using wavelet transforms and particle swarm optimization

IET Image ProcessingVolume 15, Issue 12 p. 2833-2840 ORIGINAL RESEARCH PAPEROpen Access Image resolution and contrast enhancement with optimal brightness compensation using wavelet transforms and particle swarm optimization V Hyma Lakshmi Tirumani, Corresponding Author tvhymalakshmi@gmail.com E.C.E. Department, S.R.K.R. Engineering College, Bhimavaram, Guntur, India Correspondence T.V. Hyma Lakshmi, Flat No:304. fourth floor, Sri Rama Towers, Near Rayalam Post office, Rayalam, Bhimavaram 534208, West Godavari District, Andhra Pradesh, India. Email: tvhymalakshmi@gmail.comSearch for more papers by this authorMadhu Tenneti, Education Consultancy, Bhimavaram, Andhra Pradesh, IndiaSearch for more papers by this authorCh. Srikavya K, KLEF, Andhra Pradesh, IndiaSearch for more papers by this authorSarat Kumar Kotamraju, KLEF, Andhra Pradesh, IndiaSearch for more papers by this author V Hyma Lakshmi Tirumani, Corresponding Author tvhymalakshmi@gmail.com E.C.E. Department, S.R.K.R. Engineering College, Bhimavaram, Guntur, India Correspondence T.V. Hyma Lakshmi, Flat No:304. fourth floor, Sri Rama Towers, Near Rayalam Post office, Rayalam, Bhimavaram 534208, West Godavari District, Andhra Pradesh, India. Email: tvhymalakshmi@gmail.comSearch for more papers by this authorMadhu Tenneti, Education Consultancy, Bhimavaram, Andhra Pradesh, IndiaSearch for more papers by this authorCh. Srikavya K, KLEF, Andhra Pradesh, IndiaSearch for more papers by this authorSarat Kumar Kotamraju, KLEF, Andhra Pradesh, IndiaSearch for more papers by this author First published: 21 May 2021 https://doi.org/10.1049/ipr2.12268AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onEmailFacebookTwitterLinked InRedditWechat Abstract Improving the image resolution and contrast along with uniform brightness distribution over the entire image helps to retrieve the vital realistic information necessary for human perception and interpretation. A new procedure to improve these parameters is implemented and tested. Initially, the image is super resolute using discrete wavelet transform (DWT), stationary wavelet transform (SWT) image decomposition and bicubic interpolation. The resolute image is used for contrast enhancement by using SWT and contrast limited adaptive histogram equalization (CLAHE) approach. The optimized brightness compensation technique, which uses particle swarm optimization (PSO), is applied to the resolution and contrast-enhanced image to obtain uniform brightness distribution over the entire image. The proposed approach is tested on three data sets of images and it is found that the visual results, as well as the resolution and contrast levels of the tested images, are significantly superior to subjective and objective results. 1 INTRODUCTION Image enhancement (IE) is an essential pre-processing technique to improve the quality of images and their analysis in applications like medical imaging to detect tumours [1], weather monitoring, geosciences, geographical conditions etc. [2-4]. The image can be enhanced in terms of resolution and contrast to get qualitative (best visual) and quantitative (maximum PSNR) improvement without disturbing the edges and textures [5-7]. Resolution enhancement (RE) increases the number of pixels in an image. Bicubic interpolation is widely used in RE, in which the intensity value of the unknown pixel is assigned by the weighted average of the sixteen surrounding pixels [8-10]. Further, in recent times, it is found that the wavelet transforms (WT) [11, 12] play an important role in decomposing the image into different frequency sub-bands. WT gives good time and frequency resolution for both stationary and non-stationary signals and images. Recently Hassan Demirel et al. proposed an RE technique using stationary wavelet transform (SWT) and DWT with bicubic interpolation. Due to shift-invariance, SWT improves visual results and PSNR than the previously existing methods [13]. RE improves only the number of pixels in an image. But to get the complete IE and to get better visual results, resolution enhanced image has to be contrast-enhanced. Contrast enhancement (CE) improves the intensity or brightness of the image. Histogram equalization (HE) is the basic method used in CE [14]. In a histogram equalized image, the original or input image intensity values are required to reduce the artefacts to curtail the noise enhancement. Contrast limited adaptive histogram equalization (CLAHE) is used to avoid this drawback and to get the desired histogram [15]. In CLAHE, the image is decomposed into different non-overlapping sub-blocks and then HE is applied on those sub-blocks separately. Huang Lidong et al. proposed a new method by combining the local histogram equalization (LHE) and image decomposition technique [16]. In this method, the image is decomposed into low-frequency components (LFC) and high frequency (HFC) components and then CLAHE is applied on LFC. HFC is unchanged to avoid the blurring effect on edges. Then, the weighted average is applied to the image to reduce the over brightness. This method reduces enhancement and preserves the edges [17]. If the illumination of the image is very low or very high, this type of CE cannot be applied, because of the empirical selection of brightness compensation factor 'β' and regulatory factor 'α'. Hence, we proposed a robust CE method in which the optimized brightness compensation factor 'β' and regulatory factor 'α' are obtained with the particle swarm optimization (PSO) approach [18]. This method can be applied to any type of illumination and IE process. 2 PROPOSED METHOD The proposed robust IE in resolution, contrast and optimal brightness is shown in Figure 1. The resolution enhanced image is obtained with the combination of DWT, SWT decomposition and Bi-cubic interpolation. The DWT image decomposition provides reduced computation time. In one level DWT image decomposition, the input image is passed through a series of high pass and low pass filters to obtain high frequency and low-frequency components. Daubechies 7, 9 and 12 are used as mother wavelets for three different test images empirically. The amount of detailed information in an image is obtained with high pass filters and the scale is changed by down sampling. DWT subsampling (down sampling) by a factor of '2' minimizes the samples '2' times. The image is passed through the half-band filter with impulse response h(n) resulting from a convolution operation is shown in Equations (1)–(3). x n ∗ h n = ∑ k = − ∞ ∞ ( x k − h n − k ) (1) FIGURE 1Open in figure viewerPowerPoint Block diagram of proposed method The frequencies above half of the highest frequencies are obtained from the image when the image is passed through a high pass filter. Half of the samples are removed from the image, subsampling factor being two. The subsampled image contains half the number of pixels without loss of information and reduced computational complexity. The scale is doubled in inverse filtering. The input image X(n) is allowed to pass through high pass filter g(n) and then it is passed through a low pass filter h(n). The image is decomposed into low frequency (LL, LH) and high-frequency (HL, HH) components. y high k = ∑ n x n ∗ g 2 k − n (2) y low k = ∑ n x n ∗ h 2 k − n (3)yhigh(k) is high pass filter output, ylow(k) is low pass filter output and '*' represents the convolution operation. The decomposed LL sub-band is not disturbed, whereas the other sub-band images LH, HL and HH are further enhanced with bi-cubic interpolation by a factor of 2. To determine the intensity value of the middle pixel, the weighted average of neighbouring sixteen pixels is considered. Hence, the number of pixels in the image increases; eventually increasing the resolution further. The original input image is also processed with SWT to reduce the information loss due to down sampling in DWT. The image is decomposed into LL, LH, HL and HH sub band images. SWT performs like DWT without down sampling. The sub band's size is the same as the original image. The high-frequency sub bands LH, HL and HH are combined with interpolated LH, HL and HH sub bands. These estimated sub bands and the original image are applied to inverse discrete wavelet transform for the reconstruction of the image. Resolution enhanced image is obtained with this approach. Contrast enhancement (CE) is obtained with the combination of CLAHE and SWT. The super resolute image is decomposed into low frequency and high-frequency Sub-bands using SWT. To limit the noise amplification, high-frequency components are unchanged. The low-frequency components are enhanced with CLAHE. CLAHE decomposes the image into contextual regions. Histogram equalization is applied to each region of the image separately. CLAHE alone introduces a significant drawback. An extreme black is accredited to the low range density number of the original histogram image. An extreme white is accredited to the high-end value. Between these two extreme ends, the remaining pixel values are distributed. The resolute and contrast-enhanced original image suffers from irrelevant brightness levels. By fine-tuning, the weighting parameters, brightness compensation factor β and regulatory component α, the decreased image luminance is compensated. After estimating the tuned α value, the β value is tuned to attain reasonably good PSNR. The regions of the image with low brightness are enhanced with increasing α value to attain uniform brightness over the entire image. The weighted average of the input and enhanced image are used to compensate for the over brightness [10] and is represented as I Comp = I low . M α + β . I CE One s Matrix − M α (4)where IComp = brightness compensated enhanced image Ilow = low contrast image or input image β = brightness compensation factor ICE = contrast enhanced image Onesmatrix = matrix of all ones with the same size of the input image Mα = weighing coefficients matrix with the same size of the input image Mα (i, j) = (IC (i, j))α The sizes of both Mα and IC are m × n. i = 1, 2, 3,…, m, and j = 1, 2, 3,…, n. I C i , j = I low i , j − I lowmin I lowmax − I lowmin (5) Ilowmax and Ilowmin are the highest and least intensity values of the low contrast image or input image respectively. α is a regulatory exponent. α min and α max are the extreme limits of α. 2.1 Selection of weighting parameters The image is further enhanced by varying α and β values. Here two extreme cases are observed. Those are, the original image is the same as an enhanced image when α = 0 and over brightness is attained when α is at a maximum value. Hence a systematic approach is essential for better enhancement. Particle swarm optimization (PSO) is adopted to elect the optimum β and α values to get maximum PSNR and better visual results by brightness adjustment of the resolution and contrast enhanced image. PSO is a population-random-search based algorithm used for finding minimum/maximum value of objectives. Kennedy and Eberhart first established a solution in the year 1995 and later many variants are proposed by several researchers in the optimization family [18]. Initially for electing α of an image, β is kept constant at different values and then the corresponding PSNR values and visual results are observed. Figures 2 and 3 show the corresponding graphical and pictorial results respectively for the different values of α while β is kept constant at 2. It is observed that the quality of the image and PSNR values are decaying exponentially with α value. High PSNR is attained with low α value. FIGURE 2Open in figure viewerPowerPoint Variation of PSNR with respect to α for test image1 FIGURE 3Open in figure viewerPowerPoint Test image1 (a) Original image. Contrast enhanced images at β = 2, α = 0, 0.5, 1, 1.5, 2 in (b), (c), (d), (e) and (f) respectively Afterward, the variation of PSNR is studied by varying β and by keeping α at 1.5, 3, 4.5 and 0.5 and the results are reported in Figure 4. The pattern of PSNR variation is observed by changing β from 0 to 6. Figure 4 represents the maximum PSNR value attained at a certain point in between the extreme limits of β which is quite different from the variation of α. Hence selection of β over a certain range is crucial to enhance the image properly. But these weighting parameters vary with the input image. Hence a systematic approach is required to get proper enhancement and maximum PSNR values. FIGURE 4Open in figure viewerPowerPoint Variation of PSNR with respect to β at α = 1.5, 3, 4.5, 0.5 in (a), (b), (c) and (d) respectively 2.2 Formulation of PSNR function with weighting parameters In this method, PSNR is chosen as an objective function such a way that the selected values of α and β yield maximum PSNR. However, the variations of PSNR with regulatory factor α are exponentially decaying as we obtained maximum PSNR at α_min and minimum PSNR at α_max. When α is α_min, the output image is same as the input image and when α is α_max, over brightness results in improper brightness and poor PSNR values. Instead of considering objective function with α and β, only β is considered. Therefore the overall PSNR function is formulated as PSNR = max f β (6) The range of dependent variable βϵ [β_min β_max] and α is maintained constant throughout the study irrespective of input image as optimal selection of α is mostly at 1.5. For optimal selection of β to maximize the objective function PSNR, PSO algorithm is chosen. 3 RESULTS AND DISCUSSIONS The acquired image is processed in three stages. Initially, image resolution is enhanced in stage 1 using DWT and SWT along with bi-cubic interpolation approach, then the resultant image contrast is enhanced in stage 2 with the combination of SWT, CLAHE and weighted average method. In stage3, brightness compensation is applied on the resolution and contrast enhanced image using PSO for further enhancement. Brightness compensation is achieved by selecting the optimum values of α and β. β is the decision variable for the proposed PSO assisted weighted average CLAHE-SWT method. By keeping α at 1.5, the range of β is obtained with PSO. It has been observed that the optimum 'β' value lies between 0.2 and 6. The optimum β value of 1.9 is obtained at the maximum PSNR value of 36.63 dB for the test image1 empirically as shown in Figure 5(a). Afterwards, β value of 2.758 is obtained at the maximum PSNR value of 43.67 dB for the test image1 with the proposed method as shown in Figure 5(b). FIGURE 5Open in figure viewerPowerPoint (a) Identification of optimal β empirically for the test image1. (b) Identification of optimal β in PSO of test image1 For obtaining the optimum β value, Surface searching pattern is also simulated with PSO. Figure 6 represent the search surface pattern to obtain Optimum β value at maximum PSNR value. The surface is distributed over the entire range of β values. These values are correlated with the values obtained from Figure 2. FIGURE 6Open in figure viewerPowerPoint Surface of search pattern by the PSO for brightness compensation factor β For suitable parameters identification and for proper image enhancement, the proposed methodology is applied on three more test images having different contrasts and the corresponding results are presented in Figures 7-9 respectively. The α and β values are selected by keeping one parameter constant and varying the other parameter. Figure 7(a)–(c) represents the empirically obtained response of maximum PSNR values with varying β value, keeping α at 1.5 for the test images 2, 3 and 4 respectively. Optimal weighing parameters are estimated with PSO to adjust the brightness and resultant graphs are shown in Figure 7(d)–(f) for the test images 2, 3 and 4 respectively. FIGURE 7Open in figure viewerPowerPoint Selection of suitable weighting parameters (a) β by step wise selection for test image2, (b) β by step wise selection for test image3, (c) β by step wise selection for test image4, (d) β by PSO-based selection for test image2, (e) β by PSO-based selection for test image3, (f) β by PSO-based selection for test image4 FIGURE 8Open in figure viewerPowerPoint Test image2 (a) Original image, (b) resolution and contrast enhanced image without PSO assistance, (c) resolution and contrast enhanced image with PSO assistance. Histograms of (d) input image (e) resolution and contrast enhanced image without PSO assistance. (f) Resolution and contrast enhanced image with PSO assistance FIGURE 9Open in figure viewerPowerPoint Test image4 (a) Original image (very low contrast and low resolution image), (b) resolution and contrast enhanced image without PSO assistance, (c) resolution and contrast enhanced image with PSO assistance. Histograms of (d) input image (e) resolution and contrast enhanced image without PSO assistance, (f) resolution and contrast enhanced image with PSO assistance Figures 8-10 present the pictorial results for test images 2, 3 and 4 respectively. Figures 8-10 represent the input images; Figures 8(b), 9(b) and 10(b) represent the resolution and contrast enhanced images without PSO approach. From these pictorial results in Figures 8-10, it is evident that though the image resolution and contrast are enhanced, it suffers with improper brightness regions, i.e. brightness is unevenly distributed over the entire image. The value of regulatory component α is kept constant at 1.5 by varying β values, optimal weighing parameters are estimated with PSO approach to adjust the brightness and the resultant images are shown in Figures 8-10 for test images 2, 3 and 4 respectively, their corresponding histograms are shown in Figures 8-10 their corresponding MSE, RMSE, PSNR, structural similarity index measure (SSIM) values are obtained for empirical approach and PSO approach are tabulated in Table 1. It is obvious from the results that the proposed PSO approach for identification of optimal parameters gives better objective and subjective performance than the weighing parameters elected randomly or empirically. FIGURE 10Open in figure viewerPowerPoint Test image3 (a) Original image, (b) resolution and contrast enhanced image without PSO assistance, (c) resolution and contrast enhanced image with PSO assistance. Histograms of (d) input image (e) resolution and contrast enhanced image without PSO assistance, (f) resolution and contrast enhanced image with PSO assistance TABLE 1. PSNR, MSE, RMSE SSIM values for empirical and with PSO approach for four test images α is kept at 1.5 Test image1 Test image2 Test image3 Test image4 Empirical values (without PSO assistance) β 1.9 1.9 1.1 0.5 PSNR dB 36.63 36.64 37.26 38.97 MSE 14.128 14.095 12.220 8.2429 RMSE 3.7587 3.7544 3.4957 2.8710 SSIM 0.5412 0.5713 0.5847 0.5847 Empirical values (with PSO assistance) β 1.927 1.077 0.8642 0.3929 PSNR dB 36.78 37.28 39.45 43.62 MSE 13.648 12.164 7.3804 2.8254 RMSE 3.6944 3.4877 2.7167 1.6809 SSIM 0.6122 0.6288 0.6422 0.6322 4 CONCLUSION A novel method of robust image resolution and contrast enhancement with uniform brightness distribution is proposed. Resolution enhancement is obtained by applying wavelets decomposition and bicubic interpolation. Contrast enhancement is improved with CLAHE augmented with SWT method. Brightness adjustment is obtained over the entire image with the combination of weighted average and PSO approach. The results show that the approach is reliable and robust for enhancing the quality of the image and further exhibit that even a very minute change in weighing parameters effected the subjective (PSNR values) and objective pictorial results significantly which otherwise could not be found empirically or randomly. The PSNR got improved by 0.41%, 1.73%, 5.88% and 11.93% for the test images1, 2, 3 and 4 respectively. The SSIM got improved by 13.11%, 10.06%, 9.834% and 8.12% for the test images1, 2, 3 and 4 respectively. This significant improvement in quantitative and visual results is obtained due to the precise selection of weighing parameters in the weighted average method for the optimal brightness adjustment using PSO approach. REFERENCES 1Singh, B.B., Patel, S.: Efficient medical image enhancement using CLAHE enhancement and wavelet fusion. Int. J. Comput. Appl. 167(5), 1– 5 (2017) Google Scholar 2Iqbal, M.Z., Ghafoor, A., Siddiqui, A.M.: Satellite image resolution enhancement using dual-tree complex wavelet transform and nonlocal means. IEEE Geosci. Remote Sens. Lett. 10(3), 451– 455 (2013) CrossrefWeb of Science®Google Scholar 3Hyma Laksmi, T.V., Madhu, T., Sri Kavya, K.C..: Half-band polynomial sub bands fusion and CLAHE. J. Adv. Res. Dyn. Control Syst. 10(4), 333– 338 (2018) Google Scholar 4Hyma Lakshmi, T.V., Nayak D, R.: Denoising of image and audio by using wavelet transform. Int. J. Adv. Res. Comput. Sci. 4(5), 88– 92 (2013) Google Scholar 5Hyma Lakshmi, T.V., Praveen kumar, M.: Image super resolution model using fuzzy rule. Int. J. Adv. Innovative Res. 4(7), 38- 42 (2020) Google Scholar 6Hyma Laksmi, T.V., Madhu, T., Sri Kavya, K.C., Esub Basha, S.: Novel image enhancement technique using CLAHE and wavelet transforms. Int. J. Sci. Eng. Technol. 5(6), 507– 511 (2016) Google Scholar 7Hyma Laksmi, T.V., et al.: Satellite image resolution enhancement using discrete wavelet transform and gaussian mixture model. Int. Res. J. Eng. Technol. 02(04), 95– 101 (2015) Google Scholar 8Hyma Laksmi, T.V., et al.: Satellite images resolution enhancement using non-decemated wavelet transformed and gaussian mixture model. Int. J. Appl. Eng. Res. 10(19), 40746– 40753 (2015) CrossrefGoogle Scholar 9Hyma Lakshmi, T.V., et al.: Image resolution and contrast enhancement using wavelet transforms and contrast limited adaptive histogram equalization. Int. J. Comput. Sci. Inf. Secur. 14(9), 969– 981 (2016) Google Scholar 10Prasanthi Kumari, N., et al.: Weighted transformation and wavelet transforms-based image resolution and contrast enhancement. In: Intelligent Communication, Control and Devices. Advances in Intelligent Systems and Computing, vol 624. Springer, Singapore (2018) Google Scholar 11Hasan, D., Gholamreza, A.: Discrete wavelet transform-based satellite image resolution enhancement. IEEE Trans. Geosci. Remote Sens. 49(6) 1997– 2004 (2011) CrossrefWeb of Science®Google Scholar 12Wang, X.: Moving window-based double haar wavelet transform for image processing. IEEE Trans. Image Process. 15(9), 2771– 2779 (2006) CrossrefPubMedWeb of Science®Google Scholar 13Demirel, H., Anbarjafari, G.: Image resolution enhancement by using discrete and stationary wavelet decomposition. IEEE Trans. Image Process. 20, 1458– 1460 (2011) CrossrefPubMedWeb of Science®Google Scholar 14Abdulla-al-Wadud, et al.: A dynamic histogram equalization for image contrast enhancement. IEEE Trans. Consum. Electron. 53(2), 593– 600 (2007) CrossrefWeb of Science®Google Scholar 15Wang, C., Ye, Z.: Brightness preserving histogram equalization with maximum entropy: A variational perspective. IEEE Trans. Consum. Electron. 51(4), 1326– 1334 (2005) CrossrefWeb of Science®Google Scholar 16Alex Stark, J.: Adaptive image contrast enhancement using generalizations of histogram equalization. IEEE Trans. Image Process. 9(5), 889– 896 (2000) CrossrefPubMedWeb of Science®Google Scholar 17Lidong, H., et al.: Combination of contrast limited adaptive histogram equalisation and discrete wavelet transform for image enhancement. IET Image Proc. 9(10), 908– 915 (2015) Wiley Online LibraryWeb of Science®Google Scholar 18Kennedy, J., Eberhart, R.: Particle swarm optimization. In: Proceedings of ICNN'95 - International Conference on Neural Networks. Perth, Western Australia, pp. 1942– 1948 (1995) Google Scholar Volume15, Issue12October 2021Pages 2833-2840 FiguresReferencesRelatedInformation