# Daydirex: Translating Nighttime Scenes towards Daytime Driving Experience at Night
## 2 Related Work
In this work, the model for night-to-day translation is designed based on ComboGAN [20], which is equivalent to CycleGAN for the case of two domains. Although the generator networks are identical to the ones in CycleGAN, the discriminators are modified to contain three networks specialized for different aspects of input image, blurred RGB, luminance, and gradients. They show that the proposed model improves the performance of localizing nighttime query images given a set of daytime reference images. However, night-to-day image translation is still a challenging task. When changing nighttime images to daytime ones, too dark parts of the night images often exist, resulting in poor translation quality with existing translation models. To address the problem, we propose a new approach in this paper, using already existing daytime images (e.g., street view images) as hint information for translation. Although we briefly introduce the problem and our initial idea in [5], it is not trivial to realize the approach. In this paper, we present a detailed processing pipeline and solutions to realize the idea which were not addressed previously. Also, we present in-depth evaluation results. Table 1 compares our work with the related works. To implement our idea, it is necessary to address a correspondence problem between input nighttime images acquired from a vehicle's camera and hint images acquired through the street view services. The correspondence problem is a very common problem in the field of computer vision. This is mainly caused by spatial and temporal differences between images or distortion that occurs in the process of expressing three-dimensional reality in two dimensions. In our target scenario, there are two issues, position matching and perspective matching. First, the locations where nighttime images are acquired are highly likely to be different from the locations where the corresponding street view images are acquired. To provide appropriate hint information, we need to figure out a certain area of the street view image to match the input nighttime image. Although it could be possible to apply image registration techniques, we adopt a simple method of comparing the coordinates from which the two images are acquired and calculating the distance between them. Image registration is generally used in the correspondence problem in diverse domains such as medical images [21; 22]. We expect that we can elaborate a solution to the position matching problem through image registration techniques, which we leave as future work. Second, the perspective of those two sets of images would be different since the position of the camera attached to the vehicle for acquiring street view images is different from the position of the camera to be used for our application scenario. To address this problem, perspective transformation can be used, which is a linear projection where three-dimensional objects are projected on a two-dimensional picture plane. It is used to solve the perspective problem in various domains such as road driving, industrial sites, and traffic [23; 24; 25; 26]. In our proposed solution, we apply a warping technique based on perspective transformation.