# A Low-Light Image Enhancement Method Based on Image Degradation Model and Pure Pixel Ratio Prior

Authors: Zhenfei Gu; Can Chen; Dengyin Zhang
Journal: Mathematical Problems in Engineering
 (2018)

---

## Abstract

Images captured in low-light conditions are prone to suffer from low visibility, which may further degrade the performance of most computational photography and computer vision applications. In this paper, we propose a low-light image degradation model derived from the atmospheric scattering model, which is simple but effective and robust. Then, we present a physically valid image prior named pure pixel ratio prior, which is a statistical regularity of extensive nature clear images. Based on the proposed model and the image prior, a corresponding low-light image enhancement method is also presented. In this method, we first segment the input image into scenes according to the brightness similarity and utilize a high-efficiency scene-based transmission estimation strategy rather than the traditional per-pixel fashion. Next, we refine the rough transmission map, by using a total variation smooth operator, and obtain the enhanced image accordingly. Experiments on a number of challenging nature low-light images verify the effectiveness and robustness of the proposed model, and the corresponding method can show its superiority over several state of the arts.


---

## Body

## 1. Introduction

Owing to the insufficient incident radiance received by the scene objects, images captured in the low-light environment are prone to suffer from low visibility, such as reduced contrast, faint color, and blurred scene details. This is a major problem for most applications of computational photography and computer vision which are primarily designed for high-quality inputs, including satellite imaging [1], object recognition [2], intelligent vehicles [3], etc. Therefore, unveiling the scene details hidden within the dark regions to further enhance the overall visual quality, which is often referred to as “low-light image enhancement”, is highly desired and has strong implications.Intuitively, amplifying the global pixel intensity is the simplest way to enhance the visibility of an input low-light image. Nevertheless, most low-light images are not degraded uniformly, which means the bright regions and dark regions may exist simultaneously within the same image, and the brightest regions may be overenhanced inevitably when the darkest region becomes visible. Therefore, histogram equalization [4, 5], as well as adaptive histogram equalization [6], attempted to avoid this problem by adjusting the global dynamic range. However, the balance between dark-region-recovery and bright-region-preserving is still difficult to achieve. In addition, these strategies adjust the pixel intensity structure-blindly rather than take the degradation mechanism into consideration and therefore are subject to weak robustness and limited enhancement effect.Benefitting from the Retinex theory [7], the aforementioned problem can be theoretically tackled by separating and compensating the attenuated reflectance component accordingly. Nevertheless, the reflectance component separating is fundamentally related to the illumination component estimation which is a mathematically ill-posed problem [7–9]. Early on, relevant algorithms proposed based on the illumination spatial smoothness assumption include the random walk type [7, 10], homomorphic filtering [9], Poisson equation type [11, 12]. However, one specific constraint towards the illumination component is obviously insufficiency [8]. Consequently, these methods are similar and share one common main drawback: the illumination component is prone to be inaccurately estimated. Therefore, information loss, as well as structure distortion, appears in the enhanced image inevitably. The main motivation of the subsequent techniques, such as Multiscale Retinex [13–15] and Multiscale Retinex with color restoration [16], is to improve the estimation accuracy of illumination component by utilizing a variety of complicated filtering methods. Despite the obvious progress, the inherent drawback still exists since the Retinex model lacks sufficient constraints towards the illumination component estimation.Aiming at overcoming this problem, Kimmel further proposed the variational Retinex model [8], which exploits more image mathematical statistics (prior knowledge) as additional model constraints [17–21]. For instance, [19] added the sparsity prior and the quadratic fidelity prior of the reflectance and improved the detail-preserving behavior; [20] introduced the total variation regularized term into [8] and increased the visual fidelity; [21] combined the Gaussian curvature regularization as well as the first-order differential term into [9] and guaranteed the staircase effect suppression ability. Despite the improved enhancement quality, computational complexity is sacrificed as a trade-off because the minimization problem of the variational Retinex-based models is QP with respect to the unknown image and the overall computational consumption generally depends on the model complexity.Compared with the variational Retinex-based methods, another technical clue shows its superiority with respect to computational efficiency. More concretely, Dong [25] experimentally discovered the similarity between the inverted low-light images and the hazy images and intended to remove “haze” within the inverted low-light image. After “haze” removal, the “haze-free” output was inverted again to generate the enhanced low-light image. Dong’s method is fundamentally based on the atmospheric scattering model (ASM) [26–28] as well as the DCP-based transmission estimation strategy [22]. Following this technical clue, [29] further improved the enhancement quality by introducing oversegmenting and adaptively scene-wise denoising as preprocessing for the DCP-based transmission estimation. However, the physical meaning of the “inverted low-light image enhancement model” is fundamentally hard to explain, and the enhancement effect is lacking in robustness. Although [18] theoretically connected the “inverted low-light image enhancement model” to a more physically meaningful Retinex-based enhancement methods [7], the equivalence is only valid under two strict conditions, as we will later make explicit in the next section.Aiming at addressing the main drawbacks of the existing models, we propose a low-light image degradation model which is derived from the ASM. Compared with the aforementioned models, the proposed one is simple but effective and the physical meaning is clear when describing the degradation mechanism of the low-light image. Based on the proposed model, we also present a high-efficiency low-light image enhancement method. The main concept behind the proposed method is to estimate the model coefficients by fully leveraging the scene-based potential similarity and thereby improving the computational efficiency. More concretely, we first segment the low-light image into scenes according to the brightness component similarity via thek-means clustering algorithm. Then, we propose a pure pixel ratio prior, which is a statistical regularity of extensive nature clear images. This prior, combined with the proposed model, enables us to directly estimate the rough scene transmission within each scene. Next, we introduce a total variation smooth operator to refine the rough transmission map, thereby avoiding the potential negative effects within the enhanced image. Experiments results verify that the proposed model is physically valid, and the corresponding method can show its superiority over the state of the art.


## 2. Background

### 2.1. Atmospheric Scattering Model

In some specific scenarios, light is attenuated inevitably along its original course, and this process is commonly mathematically modeled via the following atmospheric scattering model (ASM) [26–28]:(1)Ix,y=L∞·ρx,y·tx,y+L∞·1-tx,y,where x,y is the pixel index, Ix,y denotes the input image, Jx,y=L∞·ρx,y represents the corresponding enhanced image, L∞ is the global atmospheric light which describes the intensity of the atmospheric radiance, ρx,y is the pixel albedo which describes the reflectance properties of scene objects, and tx,y is the transmission which indicates the attenuation extent of radiance before it reaches the camera. The first term on the right-hand side of (1), L∞·ρx,y·tx,y, is called “direct attenuation,” which describes the image component from decayed scene-reflected radiance; the second term, L∞·1-tx,y, is called “airlight,” which indicates the image component directly from attenuated global atmospheric light. The main concept of ASM is demonstrated in Figure 1(a).Figure 1
The main concept of ASM, and the corresponding invalid example. (a) The main concept of ASM. (b) The corresponding invalid example.
(a) (b)According to (1), the enhanced image can be obtained with the knowledge of global atmospheric light and transmission, which can be expressed as follows:(2)Jx,y=L∞·ρx,y=Ix,y-L∞·1-tx,ytx,y.The ASM is rarely applied in the low-light image enhancement field due to its two inherent limitations, which can be explained as follows:( 1 ) According to (1), L∞ is a global constant, which means that the environment radiance must be uniform throughout the whole image. However, this condition cannot be fully satisfied for some images (see the dark scene within the blue box in Figure 1(a) where the global atmospheric light cannot reach directly; and the natural image shown in Figure 1(b) is obviously nonuniformly illuminated). Besides, most ASM-based global atmospheric light estimation strategies take the pixel with the highest intensity (or approximately) [22, 23, 30] as the global atmospheric light; thus some regions (especially the dark regions) tend to be darker inevitably since their true scene radiance must be weaker than the global atmospheric light [22].( 2 ) In addition, even the global constant environment radiance does exist for low-light images; L∞ should have quite low intensity (that is why the image is called low-light image). Jx,y=L∞·ρx,y must have lower intensity compared with L∞ (since ρx,y should be less than 1); thus the “clear image” must tend to be darker.In order to verify the analysis above, we process Figure1(b) using two ASM-based methods [22, 23] and demonstrate the corresponding results in Figures 2(a) and 2(b). As we can see from Figures 2(a) and 2(b), no hidden scene detail becomes visible, and the images tend to be darker inevitably. Further, we process Figure 1(b) using the same estimation methods of [22, 23] but based on the proposed model (as we will explain in Section 3) and show the enhanced images in Figures 2(c) and 2(d). It is obvious that the enhancement quality can be significantly improved.Figure 2
The model comparison. (a) The enhancement result of [22]. (b) The enhancement result of [23]. (c) The result obtained using the estimation method of [22] based on the proposed model. (c) The result obtained using the estimation method of [23] based on the proposed model.
(a) (b) (c) (d)


### 2.2. Inverted Low-Light Image Enhancement Model

Based on the observation that inverted low-light images look similar to hazy images, [25] attempted to introduce ASM into the low-light image enhancement field and proposed the “inverted low-light image enhancement model” as follows:(3)Sx,y=1-Ix,ySx,y=1-Jx,y·Tx,y+A·1-Tx,y,where Ix,y is the input low-light image, Sx,y is the corresponding inverted image, Tx,y and A are the transmissions and global atmospheric light with respect to Sx,y, respectively.Actually, the success of (3) lies in partially addressing the aforementioned limitations of ASM. This is because the image information is linearly amplified, and therefore the scene radiance is not weak anymore. Unfortunately, the physical meaning of (3) is fundamentally hard to explain. Thus, [18] attempts to connect this unrooted model to a more physically meaningful one (a specific Retinex-based method [7]). Specifically, after transmission estimation via DCP [22], the estimated transmission map of (3) can be expressed as(4)Tx,y=1-1A+maxc1-Scx,yA,where c is the RGB color space. Substituting (4) back into (3), we have(5)Jx,y=A-Sx,y1-1/A+maxc1-Scx,y/A+ϵ+1-A.As we can see, whenA=1, (5) equals a wildly used Retinex-based method [7]. Nevertheless, this equivalence relationship only exists under two strict constraints: (1) DCP [22] must hold and (2)A must strictly be 1.Apparently, the first constraint is hard to be satisfied, becauseSx,y should be bright overall (since it is inverted from a low-light image), and therefore the scene radiance for the brightest regions (which used to be the darkest regions) should be quite similar to the global atmospheric light A. However, according to [22], DCP is fundamentally invalid for the region where the scene intensity is similar to the atmospheric light.


## 2.1. Atmospheric Scattering Model

In some specific scenarios, light is attenuated inevitably along its original course, and this process is commonly mathematically modeled via the following atmospheric scattering model (ASM) [26–28]:(1)Ix,y=L∞·ρx,y·tx,y+L∞·1-tx,y,where x,y is the pixel index, Ix,y denotes the input image, Jx,y=L∞·ρx,y represents the corresponding enhanced image, L∞ is the global atmospheric light which describes the intensity of the atmospheric radiance, ρx,y is the pixel albedo which describes the reflectance properties of scene objects, and tx,y is the transmission which indicates the attenuation extent of radiance before it reaches the camera. The first term on the right-hand side of (1), L∞·ρx,y·tx,y, is called “direct attenuation,” which describes the image component from decayed scene-reflected radiance; the second term, L∞·1-tx,y, is called “airlight,” which indicates the image component directly from attenuated global atmospheric light. The main concept of ASM is demonstrated in Figure 1(a).Figure 1
The main concept of ASM, and the corresponding invalid example. (a) The main concept of ASM. (b) The corresponding invalid example.
(a) (b)According to (1), the enhanced image can be obtained with the knowledge of global atmospheric light and transmission, which can be expressed as follows:(2)Jx,y=L∞·ρx,y=Ix,y-L∞·1-tx,ytx,y.The ASM is rarely applied in the low-light image enhancement field due to its two inherent limitations, which can be explained as follows:( 1 ) According to (1), L∞ is a global constant, which means that the environment radiance must be uniform throughout the whole image. However, this condition cannot be fully satisfied for some images (see the dark scene within the blue box in Figure 1(a) where the global atmospheric light cannot reach directly; and the natural image shown in Figure 1(b) is obviously nonuniformly illuminated). Besides, most ASM-based global atmospheric light estimation strategies take the pixel with the highest intensity (or approximately) [22, 23, 30] as the global atmospheric light; thus some regions (especially the dark regions) tend to be darker inevitably since their true scene radiance must be weaker than the global atmospheric light [22].( 2 ) In addition, even the global constant environment radiance does exist for low-light images; L∞ should have quite low intensity (that is why the image is called low-light image). Jx,y=L∞·ρx,y must have lower intensity compared with L∞ (since ρx,y should be less than 1); thus the “clear image” must tend to be darker.In order to verify the analysis above, we process Figure1(b) using two ASM-based methods [22, 23] and demonstrate the corresponding results in Figures 2(a) and 2(b). As we can see from Figures 2(a) and 2(b), no hidden scene detail becomes visible, and the images tend to be darker inevitably. Further, we process Figure 1(b) using the same estimation methods of [22, 23] but based on the proposed model (as we will explain in Section 3) and show the enhanced images in Figures 2(c) and 2(d). It is obvious that the enhancement quality can be significantly improved.Figure 2
The model comparison. (a) The enhancement result of [22]. (b) The enhancement result of [23]. (c) The result obtained using the estimation method of [22] based on the proposed model. (c) The result obtained using the estimation method of [23] based on the proposed model.
(a) (b) (c) (d)


## 2.2. Inverted Low-Light Image Enhancement Model

Based on the observation that inverted low-light images look similar to hazy images, [25] attempted to introduce ASM into the low-light image enhancement field and proposed the “inverted low-light image enhancement model” as follows:(3)Sx,y=1-Ix,ySx,y=1-Jx,y·Tx,y+A·1-Tx,y,where Ix,y is the input low-light image, Sx,y is the corresponding inverted image, Tx,y and A are the transmissions and global atmospheric light with respect to Sx,y, respectively.Actually, the success of (3) lies in partially addressing the aforementioned limitations of ASM. This is because the image information is linearly amplified, and therefore the scene radiance is not weak anymore. Unfortunately, the physical meaning of (3) is fundamentally hard to explain. Thus, [18] attempts to connect this unrooted model to a more physically meaningful one (a specific Retinex-based method [7]). Specifically, after transmission estimation via DCP [22], the estimated transmission map of (3) can be expressed as(4)Tx,y=1-1A+maxc1-Scx,yA,where c is the RGB color space. Substituting (4) back into (3), we have(5)Jx,y=A-Sx,y1-1/A+maxc1-Scx,y/A+ϵ+1-A.As we can see, whenA=1, (5) equals a wildly used Retinex-based method [7]. Nevertheless, this equivalence relationship only exists under two strict constraints: (1) DCP [22] must hold and (2)A must strictly be 1.Apparently, the first constraint is hard to be satisfied, becauseSx,y should be bright overall (since it is inverted from a low-light image), and therefore the scene radiance for the brightest regions (which used to be the darkest regions) should be quite similar to the global atmospheric light A. However, according to [22], DCP is fundamentally invalid for the region where the scene intensity is similar to the atmospheric light.


## 3. Low-Light Image Degradation Model

In this work, we intend to propose a simple but more physically meaningful model derived from ASM aiming at describing the degradation mechanism of a low-light image, named as low-light image degradation model (LIDM). First, let us recall the true physical meaning of ASM (Equation (1)) [26–28], in which the term “direct attenuation” (L∞·ρx,y·tx,y) can be explained as the attenuated clear image. Therefore, L∞ can be redefined as the optimal environment light which indicates the most reasonable incident light that could enable the pixel to have the best visibility. Furthermore, considering most low-light images are nonuniformly illuminated, in order to overcome the limitation of ASM, the redefined optimal environment light should be a spatial variable rather than a global constant. Consequently, we rewrite the ASM as(6)Ix,y=Vx,y·ρx,y·tx,y+L∞·1-tx,y,where Vx,y is the redefined optimal environment light and the definition of other model coefficients are the same with ASM.Then, according to ASM, the term “airlight” is a description of the component of the environment light which directly participates in the image forming owing to the high-density atmospheric aerosols [26–28]. Thus, this term can be simply dropped for one important reason; high-density atmospheric aerosols do not exist within the whole image (if they exist, this problem should be further connected to the image dehazing field).Consequently, we further rewrite (6) by dropping the “airlight” term:(7)Ix,y=Vx,y·ρx,y·tx,y.It is worth mentioning thattx,y here describes the attenuation extent of the optimal environment light Vx,y. Therefore, (7) can be also expressed as Ix,y=Vx,y·tx,y·ρx,y. The physical meaning of (7) is obvious: the optimal environment light Vx,y decays according to the transmission tx,y and the attenuated environment light is reflected by each pixel depends on its reflectance properties ρx,y. We name (7) as the low-light image degradation model (LIDM). The primary goal of LIDM is to estimate model coefficients using the input Ix,y and further to recover the clear image Rx,y=Vx,y·ρx,y.However, according to (7), the optimal environment light Vx,y and the transmission tx,y, simultaneously dominate the visibility of a degraded image. Resolving the inherent ambiguity between the two coefficients is fundamentally ill-posed [31]. Therefore, in order to simplify the estimation procedure via LIDM, we further redefine it by considering three facts:( 1 ) Although the attenuated environment light (which is correlated with visual brightness) is a spatial variable, the local similarity property should still hold, which indicates the visual brightness within a local region is prone to be similar (it is apparently consistency with our observations on nature low-light images). Hence, we can segment Ix,y into a set of scenes according to the pixel brightness similarity, and the pixels within each scene will share approximately the same attenuated environment light (Vx,y·tx,y).( 2 ) Based on our observations on nature low-light images, we can also notice the visibility of objects with the same brightness (or within the same scene after segmentation) are tend to be attenuated equally. Besides, according to [26–28], the transmission should have the local similarity and the spatial smoothness properties. Therefore, we may assume the transmission within each scene is constant, which implies that the pixel-based variable tx,y can be redefined as a scene-constant variable ti with respect to the scene Ωi.( 3 ) After image segmentation, the attenuated environment light and as well as the transmission are both scene-constant, which further implies that the pixel-based variable Vx,y must be a scene-constant variable Vi with respect to the scene Ωi.Consequently, the proposed LIDM can be simplified as(8)Ix,y=Vi·ρx,y·ti,x,y∈Ωi,where Ωi is the scene obtained via image segmentation, Vi is the optimal scene environment light, ti is the scene transmission, and Rx,y=Vi·ρx,y represents the corresponding enhanced scene. We name (8) as the simplified LIDM. In Figure 3, we demonstrate the main concept of the proposed simplified LIDM.Figure 3
The main concept of the simplified LIDM.Comparing with the ASM, the proposed LIDM inherits the main physical meaning of ASM but is more physically meaningful by overcoming its inherent limitations when describing the degradation mechanism of the low-light image. LIDM shares the basic concept of the “inverted low-light image enhancement model” but avoids its primary drawback which is lacking in physical explanation.


## 4. Low-Light Image Enhancement via LIDM

Based on the simplified LIDM, we present a corresponding low-light image enhancement method. We first segment the low-light image into scenes according to the pixel brightness similarity usingk-means clustering algorithm [32–34]. Then, we propose a simple but powerful prior, named “pure pixel ratio prior” (PPRP), which is a statistic regularity on extensive nature clear images. The PPRP, combined with the simplified LIDM, enables us to estimate the rough scene transmission within each scene. Next, a transmission map refinement algorithm is utilized to eliminate the transmission jump phenomenon within the rough transmission map. Finally, with the refined transmission map, the enhanced low-light image can be obtained directly via the LIDM. The overview of our method is demonstrated in Figure 4.Figure 4
The overview of the proposed low-light image enhancement method.


### 4.1. Image Segmentation

According to the simplified LIDM (8), the task in this step is to segment a low-light image into scenes according to the pixel brightness similarity. This problem is similar to data clustering [35, 36], and therefore we convert this segmentation process into ak-means clustering problem [32, 33]. The clustering procedure can be expressed as(9)arg⁡minΩ⁡∑i=1k∑x,y∈ΩiBx,y-ϕi2,wherek is the cluster number, i is the index of scene Ω(i), ϕi is the relevant cluster center, and B is the brightness map of I. Thek-means clustering algorithm iteratively forms mutually exclusive clusters of a particular spatial extent by minimizing the mean square distance from each pattern to the cluster center [37]. The iteration procedure stops when a convergence criterion is satisfied, and we adopt the typical convergence criteria [33]: no (or minimal) differences after the jth iteration, that is,(10)Bjx,y-Bj-1x,y2res<ε,where Bj indicates the brightness map after jth iteration and res is the image resolution.And we setε=10-4 to terminate this procedure. In addition, after experimental comparisons on various types of 200 test images, we notice that the enhancement quality improves along with the raising cluster number but tends to be stable afterward untilk=9 (as we will explain in Section 5).In Figure5, we choose 4 challenging low-light images to demonstrate the image segmentation procedure (usingk=9). The input low-light images are depicted in the top row of Figure 5, and the corresponding segmentation results are shown in the bottom row. As we can see, the segmentation results can precisely describe the spatial distribution of the attenuated environment light, which are generally consistency with our observations.Figure 5
The low-light images and the corresponding image segmentation results. Top: low-light images. Bottom: the corresponding image segmentation results.After image segmentation, each enhanced sceneRx,y can be expressed as a function with respect to only one unknown model coefficient (scene transmission):(11)Rx,y=Vi·ρx,y=Ix,yti=fIx,y,ti,x,y∈Ωi.Obviously, according to (11), the low-light image enhancement process becomes much simpler compared with most ASM-based method, since the estimation of scene optimal environment light Vi is not necessary anymore. Thus, the primary goal is to find an optimal scene transmission ti for each scene, which enables Rx,y to have the best visibility.


### 4.2. Pure Pixel Ratio Prior

Transmission estimation is the key problem for most ASM-based image enhancement methods, and the success of this problem mainly lies in utilizing image prior knowledge [22]. Tan’s method [38] enhances the degraded images by manipulating the contrast based on the maximum contrast prior, which indicates the clear regions should have higher contrast than the plagued ones. However, this prior is unstable for the LIDM since there is no sufficient white component for the pixels within the dark scenes, and the dark pixels are prone to be darker to increase the overall scene contrast. Fattal’s method [39] is based on the prior which implies the surface shading is uncorrelated with the transmission within a local patch, but this prior is only valid for bright images. He et al. proposed the famous DCP [22]; however, it cannot work well with “inverted low-light image enhancement” as is mentioned previously. Furthermore, its estimation accuracy is unstable since the transmission estimated via DCP with different local patch sizes varies dramatically [40], and “halo” artifacts may be obvious around the abrupt depth. Zhu et al. [23] presented the color attenuation prior and therefore has the advantage of high estimation efficiency. Nevertheless, this prior is not robust for the dark scenes because the scene information is obviously insufficient. Gu et al. [41] proposed the average saturation prior which indicates the average saturation for a high-definition clear-day outdoor image tends to be around a specific value with a high probability; however, the transmission map can be only obtained when the image depth structure is known.In general, the main concept behind these image prior knowledge is to find the optimal transmission which allows the enhanced image to obey (or approach) some specific clear image feature statistic regularities (and further enables the enhanced image looks like the clear image). This thinking inspires us to experimentally observe some statistical regularities which can be applied to the low-light image enhancement.Consequently, we conduct a number of experiments on various types of about 2,000 low-light images as well as the corresponding enhanced ones (obtained using several state-of-the-art methods [6, 15, 18, 24]), respectively. Luckily, we finally notice that the “pure pixel ratio” of the low-light images and the corresponding enhanced ones, are quite different. For expression convenience, we define a term “pure pixel ratio”, which indicates the ratio of the sum of pure black and white pixels to the total pixels, and the defined “pure pixel ratio” ψ for an image I can be expressed as(12)cbx,y=1,ifIx,yRed,Blue,Green=0,0,0,elsecbx,y=0cwx,y=1ifIx,yRed,Blue,Green=1,1,1,elsecwx,y=0ψI=∑x,y∈Icbx,y+∑x,y∈Icwx,yres,where Red,Blue, and Green are theRGBcolor space indexes, respectively.According to the statistical results shown in Figure6, we can notice that most low-light images (more than 50% of the observed low-light samples, as shown in Figure 6(a)) contain a certain proportion of pure black or pure white pixels but most enhanced ones contain little (with quite low percentage of the observed enhanced samples, as shown in Figure 6(b)). In order to describe our observation more specifically, we demonstrate the statistical results using the corresponding cumulative probability distributions and observe the range of ψ(I) between 0 and 0.001 (see Figures 6(c) and 6(d)). As shown in Figure 6(c), we can only find about 19% of the observed low-light images whose “pure pixel ratio” is less than 0.001 and merely about 2% of the observed low-light images whose “pure pixel ratio” is less than 0.0001. In contrast, as shown in Figure 6(d), we can find more than 90% of the observed enhanced low-light images whose “pure pixel ratio” is less than 0.001 and about 80% of the observed enhanced low-light images whose “pure pixel ratio” is less than 0.0001.Figure 6
The statistical results of the “pure pixel ratio” on 2,000 low-light images as well as the corresponding enhanced ones. (a) and (b) the probability distributions of the “pure pixel ratio” on 2,000 low-light images and the corresponding enhanced ones. (c) and (d) the cumulative probability distributions of the “pure pixel ratio” on 2,000 low-light images and the corresponding enhanced ones.
(a) (b) (c) (d)According to the observation shown in Figure6, we infer that the “pure pixel ratio” for clear images should be less than 0.0001 with an overwhelming probability. To confirm this, we further collect 2,000 clear-day nature images (the sky region is cut manually since it inherently tends to be white and therefore may lead to inaccurate statistical results) as test samples and obtain the “pure pixel ratio” for all the samples. The statistical results are illustrated in Figure 7, and we can notice that the statistical results are almost consistency with the observation on the enhanced low-light images. Therefore, we believe that the optimal scene transmission should enable the “pure pixel ratio” of the enhanced scene to be about 0.0001, and we name this statistical regularity as “pure pixel ratio prior” (PPRP).Figure 7
The statistical results of the “pure pixel ratio” on 2,000 clear images. (a) the probability distributions of the “pure pixel ratio” on 2,000 clear images. (b) the cumulative probability distributions of the “pure pixel ratio” on 2,000 clear images.
(a) (b)


### 4.3. Scene Transmission Estimation

The proposed PPRP, combined with (11), enables us to establish the following scene transmission estimation function:(13)t~i=arg⁡minti⁡ψfIx,y,ti-0.00012,x,y∈Ωi.The core concept of (13) is to find an optimal scene transmission t~i for each scene Ωi, which will force the “pure pixel ratio” of the enhanced scene to be about 0.0001. Note that (13) is a convex function; thus we can simply obtain the optimal scene transmission using the golden section method [34, 42]. It is also worth mentioning that our method presents a scene-wise estimation strategy rather than the traditional per-pixel fashion, and therefore the processing speed will be accelerated.We estimate the rough transmission map of Figure5 via (13) and demonstrate the corresponding results in Figure 8 (see the rough transmission maps in the top row and the relevant 3D-displayed value maps in the bottom row). As we can see, the rough transmission within each scene is constant and varies between scenes. The high scene transmission value indicates the bright scene, and the low scene transmission value implies the dark scene, as we expected.Figure 8
Rough transmission maps and the corresponding 3D-displayed value maps. Top: the rough transmission maps. Bottom: the corresponding 3D-displayed value maps.However, we can also notice that the transmission jump is obvious around the scene edge and it will further lead to negative effect in the enhanced image. The abrupt transmission is mainly caused by the scene-based transmission estimation; hence, we introduce a total variation smoothing operator [24] to refine the rough transmission map as follows:(14)Et^=α2·t^-t~22+W·β2∇t^22,where t^ is the refined transmission map, t~ is the rough transmission map, and W=1-exp⁡-∇G in which G is the gradient component for each pixel. The first term is the edge-preservation term and the second term is the smoothing term.Equation (14) can be solved via a neighborhood approximate approach to improve the computational efficiency. Thus, we rewrite it as(15)Et^=α2·t^-t~2+W·β2·∑ir2-1t^-t^i2,where t^i is theith pixel within the local patch of t^ and r is the patch size. In addition, after experimental comparisons on various types of 200 test images, we notice that the enhancement quality improves along with the raising patch size r and tends to be stable afterward until r=10 (as we will explain in Section 5).Considering (15) is a quadratic function with respect to t^, we set the derivative of (15) with respect to t^ equals to zero; then we have(16)t^=α·t~+W·β·∑i=1r2-1t^iα+W·β·r2-1.Here,t^ can be obtained via solving (16) iteratively(17)t^j=α·t~+W·β·∑i=1r2-1t^ij-1α+W·β·r2-1,wherej is the iteration index. The iteration stops when t^j-t^j-122/res≤0.0001. Once the refined transmission map is known, we can put it back into (7) and generate the enhanced image.In Figure9, the refined transmission maps are shown in the top row and the relevant 3D-displayed value maps are demonstrated in the middle row. It is obvious that the transmission jump is eliminated and the important edge structures are well maintained. The bottom row of Figure 9 shows the corresponding enhanced images, and we can notice that most hidden scene details are successfully unveiled, the overall visibility is well enhanced, and no significant overenhancement/“halo” artifact appears.Figure 9
Refined transmission maps, the corresponding 3D-displayed value maps, and the relevant enhanced images. Top: the refined transmission maps. Middle: the corresponding 3D-displayed value maps. Bottom: the relevant enhanced images.


## 4.1. Image Segmentation

According to the simplified LIDM (8), the task in this step is to segment a low-light image into scenes according to the pixel brightness similarity. This problem is similar to data clustering [35, 36], and therefore we convert this segmentation process into ak-means clustering problem [32, 33]. The clustering procedure can be expressed as(9)arg⁡minΩ⁡∑i=1k∑x,y∈ΩiBx,y-ϕi2,wherek is the cluster number, i is the index of scene Ω(i), ϕi is the relevant cluster center, and B is the brightness map of I. Thek-means clustering algorithm iteratively forms mutually exclusive clusters of a particular spatial extent by minimizing the mean square distance from each pattern to the cluster center [37]. The iteration procedure stops when a convergence criterion is satisfied, and we adopt the typical convergence criteria [33]: no (or minimal) differences after the jth iteration, that is,(10)Bjx,y-Bj-1x,y2res<ε,where Bj indicates the brightness map after jth iteration and res is the image resolution.And we setε=10-4 to terminate this procedure. In addition, after experimental comparisons on various types of 200 test images, we notice that the enhancement quality improves along with the raising cluster number but tends to be stable afterward untilk=9 (as we will explain in Section 5).In Figure5, we choose 4 challenging low-light images to demonstrate the image segmentation procedure (usingk=9). The input low-light images are depicted in the top row of Figure 5, and the corresponding segmentation results are shown in the bottom row. As we can see, the segmentation results can precisely describe the spatial distribution of the attenuated environment light, which are generally consistency with our observations.Figure 5
The low-light images and the corresponding image segmentation results. Top: low-light images. Bottom: the corresponding image segmentation results.After image segmentation, each enhanced sceneRx,y can be expressed as a function with respect to only one unknown model coefficient (scene transmission):(11)Rx,y=Vi·ρx,y=Ix,yti=fIx,y,ti,x,y∈Ωi.Obviously, according to (11), the low-light image enhancement process becomes much simpler compared with most ASM-based method, since the estimation of scene optimal environment light Vi is not necessary anymore. Thus, the primary goal is to find an optimal scene transmission ti for each scene, which enables Rx,y to have the best visibility.


## 4.2. Pure Pixel Ratio Prior

Transmission estimation is the key problem for most ASM-based image enhancement methods, and the success of this problem mainly lies in utilizing image prior knowledge [22]. Tan’s method [38] enhances the degraded images by manipulating the contrast based on the maximum contrast prior, which indicates the clear regions should have higher contrast than the plagued ones. However, this prior is unstable for the LIDM since there is no sufficient white component for the pixels within the dark scenes, and the dark pixels are prone to be darker to increase the overall scene contrast. Fattal’s method [39] is based on the prior which implies the surface shading is uncorrelated with the transmission within a local patch, but this prior is only valid for bright images. He et al. proposed the famous DCP [22]; however, it cannot work well with “inverted low-light image enhancement” as is mentioned previously. Furthermore, its estimation accuracy is unstable since the transmission estimated via DCP with different local patch sizes varies dramatically [40], and “halo” artifacts may be obvious around the abrupt depth. Zhu et al. [23] presented the color attenuation prior and therefore has the advantage of high estimation efficiency. Nevertheless, this prior is not robust for the dark scenes because the scene information is obviously insufficient. Gu et al. [41] proposed the average saturation prior which indicates the average saturation for a high-definition clear-day outdoor image tends to be around a specific value with a high probability; however, the transmission map can be only obtained when the image depth structure is known.In general, the main concept behind these image prior knowledge is to find the optimal transmission which allows the enhanced image to obey (or approach) some specific clear image feature statistic regularities (and further enables the enhanced image looks like the clear image). This thinking inspires us to experimentally observe some statistical regularities which can be applied to the low-light image enhancement.Consequently, we conduct a number of experiments on various types of about 2,000 low-light images as well as the corresponding enhanced ones (obtained using several state-of-the-art methods [6, 15, 18, 24]), respectively. Luckily, we finally notice that the “pure pixel ratio” of the low-light images and the corresponding enhanced ones, are quite different. For expression convenience, we define a term “pure pixel ratio”, which indicates the ratio of the sum of pure black and white pixels to the total pixels, and the defined “pure pixel ratio” ψ for an image I can be expressed as(12)cbx,y=1,ifIx,yRed,Blue,Green=0,0,0,elsecbx,y=0cwx,y=1ifIx,yRed,Blue,Green=1,1,1,elsecwx,y=0ψI=∑x,y∈Icbx,y+∑x,y∈Icwx,yres,where Red,Blue, and Green are theRGBcolor space indexes, respectively.According to the statistical results shown in Figure6, we can notice that most low-light images (more than 50% of the observed low-light samples, as shown in Figure 6(a)) contain a certain proportion of pure black or pure white pixels but most enhanced ones contain little (with quite low percentage of the observed enhanced samples, as shown in Figure 6(b)). In order to describe our observation more specifically, we demonstrate the statistical results using the corresponding cumulative probability distributions and observe the range of ψ(I) between 0 and 0.001 (see Figures 6(c) and 6(d)). As shown in Figure 6(c), we can only find about 19% of the observed low-light images whose “pure pixel ratio” is less than 0.001 and merely about 2% of the observed low-light images whose “pure pixel ratio” is less than 0.0001. In contrast, as shown in Figure 6(d), we can find more than 90% of the observed enhanced low-light images whose “pure pixel ratio” is less than 0.001 and about 80% of the observed enhanced low-light images whose “pure pixel ratio” is less than 0.0001.Figure 6
The statistical results of the “pure pixel ratio” on 2,000 low-light images as well as the corresponding enhanced ones. (a) and (b) the probability distributions of the “pure pixel ratio” on 2,000 low-light images and the corresponding enhanced ones. (c) and (d) the cumulative probability distributions of the “pure pixel ratio” on 2,000 low-light images and the corresponding enhanced ones.
(a) (b) (c) (d)According to the observation shown in Figure6, we infer that the “pure pixel ratio” for clear images should be less than 0.0001 with an overwhelming probability. To confirm this, we further collect 2,000 clear-day nature images (the sky region is cut manually since it inherently tends to be white and therefore may lead to inaccurate statistical results) as test samples and obtain the “pure pixel ratio” for all the samples. The statistical results are illustrated in Figure 7, and we can notice that the statistical results are almost consistency with the observation on the enhanced low-light images. Therefore, we believe that the optimal scene transmission should enable the “pure pixel ratio” of the enhanced scene to be about 0.0001, and we name this statistical regularity as “pure pixel ratio prior” (PPRP).Figure 7
The statistical results of the “pure pixel ratio” on 2,000 clear images. (a) the probability distributions of the “pure pixel ratio” on 2,000 clear images. (b) the cumulative probability distributions of the “pure pixel ratio” on 2,000 clear images.
(a) (b)


## 4.3. Scene Transmission Estimation

The proposed PPRP, combined with (11), enables us to establish the following scene transmission estimation function:(13)t~i=arg⁡minti⁡ψfIx,y,ti-0.00012,x,y∈Ωi.The core concept of (13) is to find an optimal scene transmission t~i for each scene Ωi, which will force the “pure pixel ratio” of the enhanced scene to be about 0.0001. Note that (13) is a convex function; thus we can simply obtain the optimal scene transmission using the golden section method [34, 42]. It is also worth mentioning that our method presents a scene-wise estimation strategy rather than the traditional per-pixel fashion, and therefore the processing speed will be accelerated.We estimate the rough transmission map of Figure5 via (13) and demonstrate the corresponding results in Figure 8 (see the rough transmission maps in the top row and the relevant 3D-displayed value maps in the bottom row). As we can see, the rough transmission within each scene is constant and varies between scenes. The high scene transmission value indicates the bright scene, and the low scene transmission value implies the dark scene, as we expected.Figure 8
Rough transmission maps and the corresponding 3D-displayed value maps. Top: the rough transmission maps. Bottom: the corresponding 3D-displayed value maps.However, we can also notice that the transmission jump is obvious around the scene edge and it will further lead to negative effect in the enhanced image. The abrupt transmission is mainly caused by the scene-based transmission estimation; hence, we introduce a total variation smoothing operator [24] to refine the rough transmission map as follows:(14)Et^=α2·t^-t~22+W·β2∇t^22,where t^ is the refined transmission map, t~ is the rough transmission map, and W=1-exp⁡-∇G in which G is the gradient component for each pixel. The first term is the edge-preservation term and the second term is the smoothing term.Equation (14) can be solved via a neighborhood approximate approach to improve the computational efficiency. Thus, we rewrite it as(15)Et^=α2·t^-t~2+W·β2·∑ir2-1t^-t^i2,where t^i is theith pixel within the local patch of t^ and r is the patch size. In addition, after experimental comparisons on various types of 200 test images, we notice that the enhancement quality improves along with the raising patch size r and tends to be stable afterward until r=10 (as we will explain in Section 5).Considering (15) is a quadratic function with respect to t^, we set the derivative of (15) with respect to t^ equals to zero; then we have(16)t^=α·t~+W·β·∑i=1r2-1t^iα+W·β·r2-1.Here,t^ can be obtained via solving (16) iteratively(17)t^j=α·t~+W·β·∑i=1r2-1t^ij-1α+W·β·r2-1,wherej is the iteration index. The iteration stops when t^j-t^j-122/res≤0.0001. Once the refined transmission map is known, we can put it back into (7) and generate the enhanced image.In Figure9, the refined transmission maps are shown in the top row and the relevant 3D-displayed value maps are demonstrated in the middle row. It is obvious that the transmission jump is eliminated and the important edge structures are well maintained. The bottom row of Figure 9 shows the corresponding enhanced images, and we can notice that most hidden scene details are successfully unveiled, the overall visibility is well enhanced, and no significant overenhancement/“halo” artifact appears.Figure 9
Refined transmission maps, the corresponding 3D-displayed value maps, and the relevant enhanced images. Top: the refined transmission maps. Middle: the corresponding 3D-displayed value maps. Bottom: the relevant enhanced images.


## 5. Experiments

In our experiments, we implemented our method using MATLAB and approximately 0.45 seconds is required to process a600×800 pixel image using a personal computer with 2.6 GHz Intel(R) Core i5 processor and 8.0 GB RAM.In order to verify the proposed LIDM as well as the corresponding method, we select several challenging nature low-light images and conduct the qualitative and quantitative comparison with five state-of-the-art low-light image enhancement methods, including the adaptive histogram equalization (AHE) [6], Multiscale Retinex (MSR)[15], prior-based Retinex (PBR) [24], total variation Retinex (TVR) [18], and the inverted low-light image enhancement (ILIE) [25]. The parameters in our method are all demonstrated in Section 4, and the parameters in the five comparison methods are set to be optimal according to [6, 15, 18, 24, 25] for the sake of fairness.For quantitative evaluation and comparison, we adopt five extensively employed indicators, including the percentage of newly visible edgese, contrast restoration quality r, no-reference perceptual visibility indicator D, and the lightness order-error (L). According to [43], indicator e measures the ratio of edges that are newly visible after enhancement and indicator r verifies the average visibility enhancement obtained by the enhancement. According to [44], indicator D evaluates the perceptual visibility of an enhanced image. The indicator L is proposed by [45], which is used to assess the naturalness preservation ability for an image enhancement method. Generally, higher values of e and r imply better visual improvement after enhancement and lower values of D and L indicate that more scene objects are visible and better lightness order is preserved.


### 5.1. Experimental Comparisons for Clustering Number

In Section4.1, we segment a low-light image into scenes viak-means clustering algorithm. To confirm a relatively balanced clustering numberk, we conduct a large number of experiments using different values of clustering numberk. Then, we compared the enhancement effect in terms of the qualitative comparison, computational time, and five quantitatively comparison indicators (e, r, D, L, and PSNR[46]). Due to the space limitation, we demonstrate five experimental examples as shown in Figure 10.Figure 10
Five experimental examples using different clustering numbers.Figure10 shows five experimental examples obtained using different clustering numbers. As we can see, the enhancement effect is not obvious whenk equals to 1; the enhancement effect improves whenk increases from 3 to 9 and tends to be similar afterward; and whenk equals 9, most scene details can be significantly unveiled, the recovered color is natural and visually pleasing, and generally no negative effect appears.The qualitative comparison is consistent with the quantitative comparison results. As we can see from Figure11, whenk increases from 1 to 9, the values of r and L change dramatically and tend to be stable afterward whenk reaches 9, which means better visibility is obtained and good light order is maintained. The values of e, D, and PSNR are all stable whenk reaches 5, which implies that most scene details are recovered and best visual quality is achieved. Meanwhile, the computational time raises along with the increasing clustering number.Figure 11
The values of qualitative comparison indicatorse, r, D, L, and PSNR and computational time on the test examples shown in Figure 10.The observations for the demonstrated five examples are consistent with the results of more than 200 experiments. Consequently, combing all the comparison results, we assume the clustering numberk equals to 9 is a relatively balanced choice for most low-light images.


### 5.2. Experimental Comparisons for Refinement Patch Size

In Section4.3, we introduce a total variation smoothing operator to refine the rough transmission map and further present a neighborhood approximate approach to improve the relevant computational efficiency. However, the refinement effect varies according to the neighborhood patch size r which is utilized to approximate the gradient component. To confirm a relatively balanced value of patch size r, we conduct a large number of experiments and compare the enhancement quality in terms of the qualitative comparison and four quantitatively comparison indicators (e, r, D, and L). Owing to the space limitation, we list four experimental examples as follows.In Figure12, we demonstrate four experimental examples obtained using different patch size. As we can see, when r=1, negative effect ( “halo” artifacts as well as overenhancement) is obvious, especially the edges around the segmented scenes; the enhancement improvement can be noticed when r achieves 5 and keep on improving along with the increased r; and when r is greater than 10, the enhancement quality tends to be stable. The qualitative comparison is consistent with the observation on the quantitative comparison. As shown in Figure 13, when r increases from 1 to 10, the values of e and D vary dramatically and tend to be stable afterward. The values of r and L are all stable when r achieves 6. The observations for the demonstrated four examples are consistent with the results of more than 200 experiments. Consequently, we infer that the refinement patch size r=10 should be a relatively balanced choice for most low-light images.Figure 12
Four experimental examples using different refinement patch size.Figure 13
The values of qualitative comparison indicatorse, r, D, and L on the test examples shown in Figure 12.


### 5.3. Qualitative Comparisons

Figures14–19 demonstrate the qualitative comparison with five state of the arts. The input low-light images are displayed in column (a); and columns (b) to (h), from left to right, demonstrate the enhancement results of AHE [6], MSR [15], PBR [24], TVR [18], ILIE [25], and ours, respectively.Figure 14
Qualitative comparison on the low-light image with large-scale dark regions and global rich scene details. (a) Low-light image. (b) The result of AHE [6]. (c) The result of MSR [15]. (d) The result of PBR [24]. (e) The result of TVR [18]. (f) The result of ILIE [25]. (g) Our result.
(a) (b) (c) (d) (e) (f) (g)Figure 15
Qualitative comparison on the low-light image with large-scale bright regions and global rich edge details. (a) Low-light image. (b) The result of AHE [6]. (c) The result of MSR [15]. (d) The result of PBR [24]. (e) The result of TVR [18]. (f) The result of ILIE [25]. (g) Our result.
(a) (b) (c) (d) (e) (f) (g)Figure 16
Qualitative comparison on the low-light image with bright regions and significant dark regions. (a) Low-light image. (b) The result of AHE [6]. (c) The result of MSR [15]. (d) The result of PBR [24]. (e) The result of TVR [18]. (f) The result of ILIE [25]. (g) Our result.
(a) (b) (c) (d) (e) (f) (g)Figure 17
Qualitative comparison on the low-light image with bright region which contains rich scene details. (a) Low-light image. (b) The result of AHE [6]. (c) The result of MSR [15]. (d) The result of PBR [24]. (e) The result of TVR [18]. (f) The result of ILIE [25]. (g) Our result.
(a) (b) (c) (d) (e) (f) (g)Figure 18
Qualitative comparison on the low-light image with dark region which contains rich scene details. (a) Low-light image. (b) The result of AHE [6]. (c) The result of MSR [15]. (d) The result of PBR [24]. (e) The result of TVR [18]. (f) The result of ILIE [25]. (g) Our result.
(a) (b) (c) (d) (e) (f) (g)Figure 19
Qualitative comparison on the low-light image with large dark region which contains important edge details. (a) Low-light image. (b) The result of AHE [6]. (c) The result of MSR [15]. (d) The result of PBR [24]. (e) The result of TVR [18]. (f) The result of ILIE [25]. (g) Our result.
(a) (b) (c) (d) (e) (f) (g)As shown in columns (b) form Figures14–19, AHE cannot well enhance the dark regions (see the grassland in Figure 14(b), the zoom-in patch of Figure 15(b), and the cat ears in Figure 19(b)) when the obvious bright region exits. The limited enhancement ability is caused by avoiding the overexposure of the bright regions, which is a common drawback for most global enhancement methods. The enhancement effect of MSR is listed in columns (c) form Figures 14–19. MSR uses multiscale filtering to extract different frequency components from the degraded image and generate the output via components combination. However, it is unreliable to decide whether the observed pixel belongs to the scene object or the background, especially when the high frequency component is not obvious. Consequently, the scene details are blurred as shown in the zoom-in patch of Figures 14(c), 17(c), and 18(c). In addition, dark residual is obvious as we can see from the zoom-in patch of Figure 15(c). PBR is proposed based on the Retinex model as well as an image prior which is similar to the DCP and achieves fine results since the prior is generally robust. Nevertheless, the overall enhancement ability is not obvious when large-scale dark region exists (see Figures 14(d) and 19(d)) since the relevant prior is invalid for this scenario. Furthermore, dark residual is evident as shown in the zoom-in patch of Figure 15(d) owing to the inaccurate estimated illumination component. TVR estimates the rough illumination map by finding the maximum value in R, G, and B channels and refines it via a total variation model. However, owing to the inherent limitation of Retinex, the estimation accuracy of illumination component totally depends on the model constraints. Therefore, TVR suffers from local overexposure, as we can see from the zoom-in patches of Figures 14(e), 16(e), and 18(e). Besides, the ability of detail-prevention for TVR is unreliable compared with ours (see the green zoom-in patch of Figure 19(e)). Columns (f) form Figures 14–19 demonstrate the enhancement effect of ILIE. The dark residual can be noticed in the zoom-in patch of Figures 15(f) and 18(f), and block effect is obvious in the zoom-in patch of Figure 17(f). This is caused by the limitation of DCP which is fundamentally unreliable for the inverted low-light image as we explained before. Benefitting from the proposed LIDM, our method unveils most of the scene details, maintains the color fidelity, and eliminates the overexposure, with minimal “halo” artifacts.


### 5.4. Quantitative Comparisons

To quantitatively assess and rate the five state-of-the-art methods and ours, we compute four indicators (e, r, D, and L) and the computational time (C) for the enhancement effects shown in Figures 14–19 and list the corresponding results in Tables 1–5. We indicate the best value in bold and italics, and the second-best values are indicated in bold.Table 1
Value of indicatore for the enhancement results of Figures 14–19.
e AHE [6] MSR [15] PBR [24] TVR [18] ILIE [25] Ours Figure14 0.005 0.021 0.043 0.173 0.147 0.265  Figure15 0.001 0.092 0.022 0.084 0.053 0.187  Figure16 0.744 0.314 0.380 0.561 0.462 0.482  Figure17 0.259 0.104 0.080 0.082 0.116 0.139  Figure18 0.257 0.017 0.166 0.049 0.053 0.271  Figure19 0.087 0.101 0.108 0.141 0.131 0.172Table 2
Value of indicatorr for the enhancement results of Figures 14–19.
r AHE [6] MSR [15] PBR [24] TVR [18] ILIE [25] Ours Figure14 2.858 3.626 3.232 5.470 2.892 5.767  Figure15 2.346 2.070 2.639 3.683 2.274 4.007  Figure16 2.536 3.049 3.431 5.188 5.347 4.805  Figure17 2.505 2.776 2.417 4.227 2.593 2.900  Figure18 2.770 2.851 2.641 5.354 4.471 5.381  Figure19 2.656 4.272 2.099 13.711 4.679 7.035Table 3
Value of indicatorD for the enhancement results of Figures 14–19.
D AHE [6] MSR [15] PBR [24] TVR [18] ILIE [25] Ours Figure14 0.096 0.097 0.078 0.090 0.085 0.066  Figure15 0.233 0.255 0.263 0.292 0.217 0.183  Figure16 0.727 0.985 0.479 0.740 0.588 0.593  Figure17 0.400 0.559 0.585 0.293 0.615 0.250  Figure18 0.421 0.524 0.402 0.626 0.488 0.315  Figure19 0.591 0.139 0.192 0.155 0.146 0.104Table 4
Value of indicatorL for the enhancement results of Figures 14–19.
L AHE [6] MSR [15] PBR [24] TVR [18] ILIE [25] Ours Figure14 107 177 649 321 577 121  Figure15 299 280 596 464 402 307  Figure16 266 290 553 406 499 272  Figure17 328 307 694 856 311 226  Figure18 266 439 583 565 668 247  Figure19 153 234 121 234 213 218Table 5
Computational time comparison on Figures14–19.
C/s AHE [6] MSR [15] PBR [24] TVR [18] ILIE [25] Ours Figure14 0.11 0.31 1.55 0.25 0.36 0.23  Figure15 0.36 1.02 4.97 0.88 0.94 0.81  Figure16 0.23 0.76 3.16 0.55 0.57 0.44  Figure17 0.06 0.23 1.07 0.18 0.16 0.08  Figure18 0.12 0.37 1.78 0.26 0.40 0.21  Figure19 0.08 0.27 1.97 0.21 0.38 0.11According to Table1, our results achieve the best value of e for four images, which implies that most scene details can be unveiled using the proposed method. As shown in Table 2, our results yield the top values of r for three images and the second top values for two images, which are generally consistent with the results listed in Table 1. This is hard evidence that indicates that the contrast around the visible edges is improved simultaneously via the proposed method. Although our result of e for Figure 16 is relatively lower than those by AHE and TVR, the results must be balanced. As we can see in Figure 16(b), AHE dose successfully maintains the bright scene, but the visual effect is still poor since the dark scene is insufficiently enhanced. This explanation can be proved via Table 2 because AHE achieves very a low value of r for Figure 16. TVR well restores the dark regions and therefore achieves high values of both e and r; however, TVR sacrifices the bright regions since they are obviously subject to overexposure (see Figure 16(e)).The indicatorD correlates well with human judgments of image visibility [44]. As shown in Table 3, our method outperforms others for five images. This evidence verifies the outstanding visibility improvement ability of our method, and this conclusion is consistent with our observation of the qualitative comparison. Besides, since the lightness order indicates the visual naturalness of an enhanced image [45], we assess the naturalness preservation ability using indicator L and list the relevant results in Table 4. Generally, our method achieves the comparable top results with AHE, and this evidence also implies that the enhancement results obtained using our method are visually pleasing. The computational time (C) comparison results are listed in Table 5, and we can notice that the proposed method achieves the second-best results for all the six images. As aforementioned, AHE adjusts the global dynamic range structure-blindly, and therefore has the highest efficiency; however, the robustness and effectiveness are sacrificed inevitably. The proposed method achieves better enhancement quality compared with AHE and shows its superiority with respect to the efficiency over all the other model-based methods. We attribute this to the robustness of the proposed LIDM and the effectiveness of the corresponding enhancement method.


## 5.1. Experimental Comparisons for Clustering Number

In Section4.1, we segment a low-light image into scenes viak-means clustering algorithm. To confirm a relatively balanced clustering numberk, we conduct a large number of experiments using different values of clustering numberk. Then, we compared the enhancement effect in terms of the qualitative comparison, computational time, and five quantitatively comparison indicators (e, r, D, L, and PSNR[46]). Due to the space limitation, we demonstrate five experimental examples as shown in Figure 10.Figure 10
Five experimental examples using different clustering numbers.Figure10 shows five experimental examples obtained using different clustering numbers. As we can see, the enhancement effect is not obvious whenk equals to 1; the enhancement effect improves whenk increases from 3 to 9 and tends to be similar afterward; and whenk equals 9, most scene details can be significantly unveiled, the recovered color is natural and visually pleasing, and generally no negative effect appears.The qualitative comparison is consistent with the quantitative comparison results. As we can see from Figure11, whenk increases from 1 to 9, the values of r and L change dramatically and tend to be stable afterward whenk reaches 9, which means better visibility is obtained and good light order is maintained. The values of e, D, and PSNR are all stable whenk reaches 5, which implies that most scene details are recovered and best visual quality is achieved. Meanwhile, the computational time raises along with the increasing clustering number.Figure 11
The values of qualitative comparison indicatorse, r, D, L, and PSNR and computational time on the test examples shown in Figure 10.The observations for the demonstrated five examples are consistent with the results of more than 200 experiments. Consequently, combing all the comparison results, we assume the clustering numberk equals to 9 is a relatively balanced choice for most low-light images.


## 5.2. Experimental Comparisons for Refinement Patch Size

In Section4.3, we introduce a total variation smoothing operator to refine the rough transmission map and further present a neighborhood approximate approach to improve the relevant computational efficiency. However, the refinement effect varies according to the neighborhood patch size r which is utilized to approximate the gradient component. To confirm a relatively balanced value of patch size r, we conduct a large number of experiments and compare the enhancement quality in terms of the qualitative comparison and four quantitatively comparison indicators (e, r, D, and L). Owing to the space limitation, we list four experimental examples as follows.In Figure12, we demonstrate four experimental examples obtained using different patch size. As we can see, when r=1, negative effect ( “halo” artifacts as well as overenhancement) is obvious, especially the edges around the segmented scenes; the enhancement improvement can be noticed when r achieves 5 and keep on improving along with the increased r; and when r is greater than 10, the enhancement quality tends to be stable. The qualitative comparison is consistent with the observation on the quantitative comparison. As shown in Figure 13, when r increases from 1 to 10, the values of e and D vary dramatically and tend to be stable afterward. The values of r and L are all stable when r achieves 6. The observations for the demonstrated four examples are consistent with the results of more than 200 experiments. Consequently, we infer that the refinement patch size r=10 should be a relatively balanced choice for most low-light images.Figure 12
Four experimental examples using different refinement patch size.Figure 13
The values of qualitative comparison indicatorse, r, D, and L on the test examples shown in Figure 12.


## 5.3. Qualitative Comparisons

Figures14–19 demonstrate the qualitative comparison with five state of the arts. The input low-light images are displayed in column (a); and columns (b) to (h), from left to right, demonstrate the enhancement results of AHE [6], MSR [15], PBR [24], TVR [18], ILIE [25], and ours, respectively.Figure 14
Qualitative comparison on the low-light image with large-scale dark regions and global rich scene details. (a) Low-light image. (b) The result of AHE [6]. (c) The result of MSR [15]. (d) The result of PBR [24]. (e) The result of TVR [18]. (f) The result of ILIE [25]. (g) Our result.
(a) (b) (c) (d) (e) (f) (g)Figure 15
Qualitative comparison on the low-light image with large-scale bright regions and global rich edge details. (a) Low-light image. (b) The result of AHE [6]. (c) The result of MSR [15]. (d) The result of PBR [24]. (e) The result of TVR [18]. (f) The result of ILIE [25]. (g) Our result.
(a) (b) (c) (d) (e) (f) (g)Figure 16
Qualitative comparison on the low-light image with bright regions and significant dark regions. (a) Low-light image. (b) The result of AHE [6]. (c) The result of MSR [15]. (d) The result of PBR [24]. (e) The result of TVR [18]. (f) The result of ILIE [25]. (g) Our result.
(a) (b) (c) (d) (e) (f) (g)Figure 17
Qualitative comparison on the low-light image with bright region which contains rich scene details. (a) Low-light image. (b) The result of AHE [6]. (c) The result of MSR [15]. (d) The result of PBR [24]. (e) The result of TVR [18]. (f) The result of ILIE [25]. (g) Our result.
(a) (b) (c) (d) (e) (f) (g)Figure 18
Qualitative comparison on the low-light image with dark region which contains rich scene details. (a) Low-light image. (b) The result of AHE [6]. (c) The result of MSR [15]. (d) The result of PBR [24]. (e) The result of TVR [18]. (f) The result of ILIE [25]. (g) Our result.
(a) (b) (c) (d) (e) (f) (g)Figure 19
Qualitative comparison on the low-light image with large dark region which contains important edge details. (a) Low-light image. (b) The result of AHE [6]. (c) The result of MSR [15]. (d) The result of PBR [24]. (e) The result of TVR [18]. (f) The result of ILIE [25]. (g) Our result.
(a) (b) (c) (d) (e) (f) (g)As shown in columns (b) form Figures14–19, AHE cannot well enhance the dark regions (see the grassland in Figure 14(b), the zoom-in patch of Figure 15(b), and the cat ears in Figure 19(b)) when the obvious bright region exits. The limited enhancement ability is caused by avoiding the overexposure of the bright regions, which is a common drawback for most global enhancement methods. The enhancement effect of MSR is listed in columns (c) form Figures 14–19. MSR uses multiscale filtering to extract different frequency components from the degraded image and generate the output via components combination. However, it is unreliable to decide whether the observed pixel belongs to the scene object or the background, especially when the high frequency component is not obvious. Consequently, the scene details are blurred as shown in the zoom-in patch of Figures 14(c), 17(c), and 18(c). In addition, dark residual is obvious as we can see from the zoom-in patch of Figure 15(c). PBR is proposed based on the Retinex model as well as an image prior which is similar to the DCP and achieves fine results since the prior is generally robust. Nevertheless, the overall enhancement ability is not obvious when large-scale dark region exists (see Figures 14(d) and 19(d)) since the relevant prior is invalid for this scenario. Furthermore, dark residual is evident as shown in the zoom-in patch of Figure 15(d) owing to the inaccurate estimated illumination component. TVR estimates the rough illumination map by finding the maximum value in R, G, and B channels and refines it via a total variation model. However, owing to the inherent limitation of Retinex, the estimation accuracy of illumination component totally depends on the model constraints. Therefore, TVR suffers from local overexposure, as we can see from the zoom-in patches of Figures 14(e), 16(e), and 18(e). Besides, the ability of detail-prevention for TVR is unreliable compared with ours (see the green zoom-in patch of Figure 19(e)). Columns (f) form Figures 14–19 demonstrate the enhancement effect of ILIE. The dark residual can be noticed in the zoom-in patch of Figures 15(f) and 18(f), and block effect is obvious in the zoom-in patch of Figure 17(f). This is caused by the limitation of DCP which is fundamentally unreliable for the inverted low-light image as we explained before. Benefitting from the proposed LIDM, our method unveils most of the scene details, maintains the color fidelity, and eliminates the overexposure, with minimal “halo” artifacts.


## 5.4. Quantitative Comparisons

To quantitatively assess and rate the five state-of-the-art methods and ours, we compute four indicators (e, r, D, and L) and the computational time (C) for the enhancement effects shown in Figures 14–19 and list the corresponding results in Tables 1–5. We indicate the best value in bold and italics, and the second-best values are indicated in bold.Table 1
Value of indicatore for the enhancement results of Figures 14–19.
e AHE [6] MSR [15] PBR [24] TVR [18] ILIE [25] Ours Figure14 0.005 0.021 0.043 0.173 0.147 0.265  Figure15 0.001 0.092 0.022 0.084 0.053 0.187  Figure16 0.744 0.314 0.380 0.561 0.462 0.482  Figure17 0.259 0.104 0.080 0.082 0.116 0.139  Figure18 0.257 0.017 0.166 0.049 0.053 0.271  Figure19 0.087 0.101 0.108 0.141 0.131 0.172Table 2
Value of indicatorr for the enhancement results of Figures 14–19.
r AHE [6] MSR [15] PBR [24] TVR [18] ILIE [25] Ours Figure14 2.858 3.626 3.232 5.470 2.892 5.767  Figure15 2.346 2.070 2.639 3.683 2.274 4.007  Figure16 2.536 3.049 3.431 5.188 5.347 4.805  Figure17 2.505 2.776 2.417 4.227 2.593 2.900  Figure18 2.770 2.851 2.641 5.354 4.471 5.381  Figure19 2.656 4.272 2.099 13.711 4.679 7.035Table 3
Value of indicatorD for the enhancement results of Figures 14–19.
D AHE [6] MSR [15] PBR [24] TVR [18] ILIE [25] Ours Figure14 0.096 0.097 0.078 0.090 0.085 0.066  Figure15 0.233 0.255 0.263 0.292 0.217 0.183  Figure16 0.727 0.985 0.479 0.740 0.588 0.593  Figure17 0.400 0.559 0.585 0.293 0.615 0.250  Figure18 0.421 0.524 0.402 0.626 0.488 0.315  Figure19 0.591 0.139 0.192 0.155 0.146 0.104Table 4
Value of indicatorL for the enhancement results of Figures 14–19.
L AHE [6] MSR [15] PBR [24] TVR [18] ILIE [25] Ours Figure14 107 177 649 321 577 121  Figure15 299 280 596 464 402 307  Figure16 266 290 553 406 499 272  Figure17 328 307 694 856 311 226  Figure18 266 439 583 565 668 247  Figure19 153 234 121 234 213 218Table 5
Computational time comparison on Figures14–19.
C/s AHE [6] MSR [15] PBR [24] TVR [18] ILIE [25] Ours Figure14 0.11 0.31 1.55 0.25 0.36 0.23  Figure15 0.36 1.02 4.97 0.88 0.94 0.81  Figure16 0.23 0.76 3.16 0.55 0.57 0.44  Figure17 0.06 0.23 1.07 0.18 0.16 0.08  Figure18 0.12 0.37 1.78 0.26 0.40 0.21  Figure19 0.08 0.27 1.97 0.21 0.38 0.11According to Table1, our results achieve the best value of e for four images, which implies that most scene details can be unveiled using the proposed method. As shown in Table 2, our results yield the top values of r for three images and the second top values for two images, which are generally consistent with the results listed in Table 1. This is hard evidence that indicates that the contrast around the visible edges is improved simultaneously via the proposed method. Although our result of e for Figure 16 is relatively lower than those by AHE and TVR, the results must be balanced. As we can see in Figure 16(b), AHE dose successfully maintains the bright scene, but the visual effect is still poor since the dark scene is insufficiently enhanced. This explanation can be proved via Table 2 because AHE achieves very a low value of r for Figure 16. TVR well restores the dark regions and therefore achieves high values of both e and r; however, TVR sacrifices the bright regions since they are obviously subject to overexposure (see Figure 16(e)).The indicatorD correlates well with human judgments of image visibility [44]. As shown in Table 3, our method outperforms others for five images. This evidence verifies the outstanding visibility improvement ability of our method, and this conclusion is consistent with our observation of the qualitative comparison. Besides, since the lightness order indicates the visual naturalness of an enhanced image [45], we assess the naturalness preservation ability using indicator L and list the relevant results in Table 4. Generally, our method achieves the comparable top results with AHE, and this evidence also implies that the enhancement results obtained using our method are visually pleasing. The computational time (C) comparison results are listed in Table 5, and we can notice that the proposed method achieves the second-best results for all the six images. As aforementioned, AHE adjusts the global dynamic range structure-blindly, and therefore has the highest efficiency; however, the robustness and effectiveness are sacrificed inevitably. The proposed method achieves better enhancement quality compared with AHE and shows its superiority with respect to the efficiency over all the other model-based methods. We attribute this to the robustness of the proposed LIDM and the effectiveness of the corresponding enhancement method.


## 6. Discussions and Conclusions

In this paper, we have proposed a simple but physically valid low-light image degradation model (LIDM) which is derived from the atmospheric scattering model. The proposed model inherits the main physical meaning of ASM, shares the basic concept of the “inverted low-light image enhancement model,” and overcomes their inherent limitations respectively. Then, we proposed an image prior named the pure pixel ratio prior based on the statistical regularity of extensive nature clear images. Next, based on the proposed model as well as the pure pixel ratio prior, we presented a high-efficiency low-light image enhancement method by fully leveraging the scene-based potential similarity. Experimental results verify the robustness of the proposed model and the effectiveness of the corresponding low-light image enhancement method.



---

*Source: 8178109-2018-07-16.xml*