# Retinex-Based Relighting for Night Photography
## 1 Introduction  
Lighting up buildings is a form of entertainment that adds color to cities. This lighting can be physically changed, but it is not easy to change the lighting on a building after taking a photograph of it. After the image is captured, photographers can change the lighting of the photo by using photo-editing applications, such as Adobe Photoshop. However, creating illuminated photography manually is a time-consuming and challenging task. In this paper, we automatically generate and change the illuminated light in an image based on the Retinex theory [1; 2; 3], which is based on the human visual system (HVS). Generally, colors in images are determined by interactions between lights and illuminated surfaces. This interaction depends on the reflectivity and shape of the subject. Therefore, we should analyze this information to create natural lighting; however, understanding this information from photographs alone is a complex problem. Retinex theory shows that HVS does not perceive absolute brightness but rather relative brightness, capturing the brightness perceived simultaneously with the ambient brightness. There are various multimedia applications based on this relation: low-light enhancement [4; 5; 6; 7; 8; 9; 10], high-dynamic range imaging [11], haze removal [12; 13], underwater image enhancement [14], shadow removal [15], denoising [16], medical imaging [17; 18], and face recognition [19]. Low-light enhancement is the application most related to the proposed method. For the enhancement of low-light images, one method is to emphasize very dark areas by accurately separating the illumination from the reflected light. Because of the multiplicative nature of the illumination and reflectance relationship, poor estimation accuracy for one will result in worse accuracy for the other. Recent approaches [5; 6; 7; 8; 20] solve this problem using complex solutions. While early studies for low-light enhancement are computationally lightweight, the reflectance component is directly separated from the illumination and regarded as the enhanced output, which often causes over-enhancement [4]. Current algorithms [5; 6; 7; 8; 20] estimate the illumination and reflectance components by adding theprior constraints, and then they obtain the enhanced low-light image by multiplying the reflectance component with the adjusted illumination component. Our research target is relighting nighttime skyline shots as a new multimedia application using Retinex theory. The difference in our approach from low-light enhancement is the dynamic range required for the final image. Low-light enhancement requires a wider dynamic range to make dark areas visible, while our application only requires a narrow dynamic range since only bright areas will change illumination, not dark areas, limiting the number of re-lit regions. Thus, the transformation of illuminated intensity can be restricted to a linear transformation instead of a non-linear one. The final image can be directly transformed with the linear transformation from the input image and illuminating the color map in the Lab color space. The additional illuminating color map in the final image is artificial; thus, the map does not require an estimation of the original physical illumination. Our method estimates only perceptual reflectance, which indicates the additional illuminating location. Only a rough shape, such as an edge, is needed to estimate reflectance; thus, our approach is lightweight and does not cause significant degradation. Our approach has three steps. First, we generate the reflectance field using single-scale Retinex (SSR) [21], and then the reflectance is convoluted by min-filtering and joint bilateral filtering [22; 23]. All filters can be a O(1) constant-time property for each filtering radius: min-filtering [24], Gaussian filtering [25], and joint bilateral filtering [26].