DOI: 10.14457/tu.the.2020.999
title: Local detail improvement for multi-exposure fusion using pyramid-based blending method

description: In the past decades, generating a high dynamic range image (HDRI) has been a favorite and challenging topic in creating a resulting image that can display a broader range of luminance levels. At the early stage, an acquisition pipeline of HDRI is created from a set of different exposure images, which can be ranged from the darkest to the brightest. The function of camera response (CRF) and the information about exposure times are used to combine each exposure image's intensity linearly. However, the method to obtain the camera response is not that easy because the researchers need to calculate the camera response every time they change the capturing devices or adjust the new parameters, e.g., exposure time, ISO, etc. Moreover, after acquiring the HDRI, the researchers need to compress the range of HDRI. The standard display device in the commercial products and low dynamic range display devices is mostly shown in 8 bits/color channel. The process of compressing the range of HDRI is so-called "Tone mapping". It can notice that the method to create and display the HDRI is quite difficult and need many steps to process. Some researchers introduce another technique to create the HDRI for displaying in the standard display devices to overcome those problems. In this case, the HDRI can be generated by measuring the salient areas or the pixels along the input image sequence and merging them. These techniques are called "the multiple-exposure fusion/blending". For example, Goshtasby et al. and Varkonyi-Koczy et al. proposed the block-based approach, where each block has the same side to measure and select the most salient areas along the sequence of input images. The algorithm's performance depends on the size of the block and the width of the blending function. Jo and his colleagues presented the cluster-based method with the bilateral filter. The algorithm can achieve a high-quality image. However, the computational time is very high due to the computational cost of the bilateral filter. Mertens et al. created the quality measurements based on the global function represent as the weighted map. Then, the pyramid-based blending is used to fuse and reconstruct the resulting image. The algorithm can immediately produce a well-exposed image for displaying on standard devices. However, it cannot achieve the local information due to pyramid-based fusion limitations at the pyramid's high levels. In this work, we introduce two versions of the pyramid-based blending method, which try to improve the local information while keeping the resulting image's perceptual quality. The first version is based on compressing some high RGB values that affect the number of pyramid-based blending levels. We use the local exposure-weight function's benefit with the symmetric property to assign the fewer weights, either dark or bright parts, to compress the high RGB values selected. We also investigate the global relationship of the intensities of the input images in the dataset to decide which RGB values should be suppressed. We call this method the 'DBLP-MEF'. The second version is based on cascading two pyramid-based blending modules. We turn a drawback of the conventional pyramid-based blending method to be beneficial. Based on our investigation, we found that some of the resulting images obtained from different levels of the conventional pyramid-based blending method contain lots of information. Some can create good quality in terms of image contrast. By combining those images, we can get a better result. We use this investigation to generate the second version. In this method, it consists of two steps: generating the virtual photograph and two-layer pyramid-based blending. In some cases, using only the input images is not enough to create a good resulting-image. Therefore, the first step is required to check whether we should generate the virtual photograph and add it to the input set. If the virtual photograph is to be generated, this step also determines the virtual photograph's average intensity. The second step blends the images in the input set. We call this second method, the '2LP-MEF'. The result images obtained from both versions of the proposed method can improve the local information compared with the conventional method proposed by Merten et al. the Image Quality Assessment measures both versions (IQA) and Detail Preserving Assessment (DPA) to evaluate the perceptual quality and the detail quality, respectively. Typical methods of multi-exposures fusion are also used in the comparison. The second version gets a high score in IQA and DPA compared with other evaluation methods. In conclusion, our proposed methods can solve the trade-off between preserving local areas and the quality in terms of smoothness boundaries. However, in some cases, the first version, DBLP-MEF, cannot generate the preferable resulting image or causes the halo effect in the resulting image. These problems come from the step of compressing the luminance or intensity among the sequence of input images. The second version, 2LP-MEF, can overcome the drawback of the DBLP-MEF in terms of the image contrast. However, in some cases, the 2LP-MEF has a color correction problem, especially in the actual dark areas. The problem comes from computing the weight function, in which the 2LP-MEF tries to improve details by identifying any dark area as an unimportant part of the blending process.