# Retinex-Based Relighting for Night Photography
## 3 Proposed Method
### Retinex-Based Image Relighting  
The output image \(\mathbf{J}:\mathcal{S}\mapsto\mathcal{R}\) consists of luminance and reflectance factors, and our retouching aims to change the image's illumination; thus, we change the luminance \(\mathbf{L}\) by a luminance remapping function \(M\) while keeping the reflectance \(\mathbf{R}\):  
\[\mathbf{J}=M(\mathbf{L})\odot\mathbf{R},\quad\text{where}\quad M(\mathbf{L})=(l+k\mathbf{F})\odot \mathbf{L}. \tag{13}\]  
The function \(M\) is based on the illuminated map \(\mathbf{F}\) computed in the previous section. Equation (13) shows that the lighting map boosts the luminance \(\mathbf{F}\) by multiplying a constant scalar factor \(k\), linearly darkening the factor \(l\leq 1\). When \(l=0.5\), the default lightness is half. Plugging in the Retinex relation of Equation (1), we can simplify Equation (13) as follows:  
\[\mathbf{J}=(l+k\mathbf{F})\odot\mathbf{L}\odot\mathbf{R}=(l+k\mathbf{F})\odot\mathbf{L}\odot\frac\mathbf{{f}}{\mathbf{L}}=(l+k\mathbf{F})\odot\mathbf{f}. \tag{14}\]  
In our condition, we assume that illumination signals change linear properties; thus, the illumination component \(\mathbf{L}\) can be removed in the final representation since the illumination components of \(\mathbf{L}\) are canceled, i.e., there is no illumination \(\mathbf{L}\) and reflectance \(\mathbf{R}\) in  
Figure 7: Visualizing result of (11). The white area is the set \(Q_{\mathbf{p}}\). (14). If we change \(L\) using a non-linear function, e.g., Gamma correction, the luminance component is required to recover the final image \(J\). In this case, we should consider the color distortion problem [4], as in low-light enhancement. The parameters \(l\) and \(k\) have essential roles in Retinex theory. Based on Retinex theory, human perception is affected by the surrounding colors. For example, in Figure 8, the two left gray backgrounds and the two right blue backgrounds, the RGB of the orange on the left side of each pair is \((231,153,99)\) and that of the orange on the right side is \((207,125,70)\). In the images with blue backgrounds, the actual brightness of the oranges on the left and right sides is different, but they appear to be the same color. On the other hand, on the gray backgrounds, the oranges appear to have a greater difference in brightness than on the blue backgrounds, making it difficult to see the same color. This effect plays an important role in Retinex-based image processing. The dynamic range of an output image on display is generally 0 to 255. Therefore, when light is added to the original image, for example, the green light may not appear green due to the upper limit of the dynamic range (Figure 9a,d). To eliminate this effect, we reduce the brightness of the surroundings to make them appear relatively brighter (Figure 9b,c,e). Figure 9b,e are darkened in the RGB color space. Figure 9b,e are darkened in the Lab color space. In this paper, we used the second method and controlled for it using \(l\) of Equation (14). Our human senses are also affected by the colors of our surroundings. In other words, the surrounding bright colors affect the new color to be added. Figure 10 shows the simulated example of the effect. The target pink colors  
Figure 8: Effect of surrounding luminance values and colors on the brightness of target color. A blue and gray background pair, with the orange RGB value on the left (231,153,99) and on the right (207,125,70), respectively. Figure 9: Appearance when illuminated with awareness of dynamic range. (**a,d**): The luminance value is added to the original image. (**b,e**): The luminance value is added after darkening the original image by reducing RGB values (\(l=0.75\)). (**c,f**): The luminance value is added to the RGB image that is converted to Lab color space (\(l_{l}=l_{a}=l_{b}=0.75\)). however, these are affected by surrounding colors, and thus, these have different values. For example, it looks orange when the target color on the right-most side is enclosed by yellow. Therefore, we control the color using the Lab color space.