Lookup Table meets Local Laplacian Filter: Pyramid Reconstruction Network for Tone Mapping
Authors: Feng Zhang
Ming Tian
Zhiqiang Li
Bin Xu
Qingbo Lu
Changxin Gao
Nong Sang
Venue: NeurIPS 2023 poster (2023)
Keywords: tone mapping; learnable local laplacian filter; laplacian pyramid; 3D lookup table

Abstract: Tone mapping aims to convert high dynamic range (HDR) images to low dynamic range (LDR) representations, a critical task in the camera imaging pipeline. In recent years, 3-Dimensional LookUp Table (3D LUT) based methods have gained attention due to their ability to strike a favorable balance between enhancement performance and computational efficiency. However, these methods often fail to deliver satisfactory results in local areas since the look-up table is a global operator for tone mapping, which works based on pixel values and fails to incorporate crucial local information. To this end, this paper aims to address this issue by exploring a novel strategy that integrates global and local operators by utilizing closed-form Laplacian pyramid decomposition and reconstruction. Specifically, we employ image-adaptive 3D LUTs to manipulate the tone in the low-frequency image by leveraging the specific characteristics of the frequency information. Furthermore, we utilize local Laplacian filters to refine the edge details in the high-frequency components in an adaptive manner. Local Laplacian filters are widely used to preserve edge details in photographs, but their conventional usage involves manual tuning and fixed implementation within camera imaging pipelines or photo editing tools. We propose to learn parameter value maps progressively for local Laplacian filters from annotated data using a lightweight network. Our model achieves simultaneous global tone manipulation and local edge detail preservation in an end-to-end manner. Extensive experimental results on two benchmark datasets demonstrate that the proposed method performs favorably against state-of-the-art methods.

Lookup Table meets Local Laplacian Filter: Pyramid Reconstruction Network for Tone Mapping
Feng Zhang
Ming Tian Zhiqiang Li 2 Bin Xu
Qingho Lu
Changxin Gao 1,3x Nong Sang
National Key Laboratory of Multispectral Information Intelligent Processing Technology, School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, DJI Technology Co., Ltd, 3 Hubei Key Laboratory of Brain-inspired Intelligent Systems {fengzaanaaia, tianming, cgao, nsang) ehust.edu..ct... en, (mila. xu, cristopher 11, Hangbboluuodd com com
Abstract
Tone mapping aims to convert high dynamic range (HDR) images to low dynamic range (LDR) representations, critical task in the camera imaging pipeline. In recent years, 3-Dimensional Look-Up Table (3D LUTT) based methods have gained attention due their ability to strike a favorable balance between enhancement performance and computational efficiency. However, these methods often fail to deliver satisfactory results in local areas since the look-up table a global operator for tone mapping, which works based on pixel values and fails to incorporate crucial local information. this end, this paper aims to address this issue by exploring a novel strategy that integrates global and local operators by utilizing closed form Laplacian pyramid decomposition and reconstruction. Specifically, we employ anageead 3D LUTs to manipulate the tone in the low-frequency image by leveraging the specific characteristics the frequency information. Furthermore, utilize local Laplacian filters refine the edge details in the figh-frequency components in an adaptive manner. Local Laplacian filters are widely used preserve edge details photographs, but their conventional usage involves manual tuning and fixed implementation within camera imaging pipelines or photo editing tools. We propose to learn parameter value maps progressively for local Laplacian filters from annotated data using a lightweight network. Our model achieves simultancous global end-to-end manner. Extensive experimental results on two benchmark datasets demonstrate that the proposed method performs favorably against state-of-the-ar methods.
manipulation and local edge detail preservation in an
Introduction
Modern cameras, despite their advanced and sophisticated sensors, are limited in their ability to capture the same level of detail the human eye in given scene. In order to capture more detail, high dynamic range (HDR) imaging techniques [25, 2] have been developed to convey a wider range of contrasts and luminance values than conventional low dynamic range (LDR) imaging. However, most modern graphics display devices have a limited dynamic range that is inadequate to reproduce the full range of light intensities present in natural scenes. To tackle this issue, tone mapping techniques [28, 34, 9] have been proposed to render high-contrast scene radiance to the
'Changxin Gao is corresponding author, email: egao@ hust.cdu.cn
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

displayable range while preserving the image details and color appearance important to appreciate the original scene content.
Traditional tone mapping operators can be classified according to their processing as global or local. Global operators [11, 34, 7, 33, 21] map each pixel according to its global characteristics, irrespective of its spatial localization. This approach entails calculating a single matching luminance value for the entire image. As result, the processing time is considerably reduced, but the resulting image may exhibit fewer details. In contrast, local operators [6. 8, 10, 24, 29] consider the spatial localization of each pixel within the image and process them accordingly. In essence, this method calculates the luminance adaptation for each pixel based on its specific position. Consequently, the resulting image becomes more visually accessible to the human eye and exhibits enhanced details, albeit at the expense of longer processing times. However, these traditional operators often require manual tuning by experienced engineers, which can be cumbersome since evaluating results necessitates testing across various scenes. Although system contributions have aimed to simplify the implementation executables [32, 15, 27], they still necessitate programming expertise, incur runtime costs that escalate with pipeline complexity, and are only applicable when the source code for the filters is available. Therefore, seeking an automatic strategy for HDR image tone mapping is of great interest.
of
In recent years, there have been notable advancements in learning dearning-based based automatic enhancement methods [37, 12, 5, 30, 16, 35, 19]. thanks to the rapid development of deep learning techniques [22]. Many of these methods focus on learning a dense aixe--topixe mapping between input high dynamic range (HDR) and output low dynamic range (LDR) image pairs. Alternatively, they predict pixel-wise transformations to map the input HDR image. However, most previous studies involve a substantial computational burden that exhibits a linear growth pattern in tandem with the dimensions of the input image.
To simultaneously improve the quality and efficiency of learning- based methods, hybrid methods [17, 41, 38, 36, 39] have emerged that combine the utilization of image priors from traditional operators with the integration of multi-level features within deep learning- of-the-art performance. Among these methods. Zeng et al. [38] proposes novel image-adaptive 3- 3-Dimensional Look-Up Table (3D LUT) based approach. which exhibits favorable characteristics such as superior image quality, efficient computational processing, and minimal memory utilization. However, as indicated by the authors, utilizing the global (spatially uniform) tone mapping operators, such as the 3D look-up tables, may produce less satisfactory results in local areas. Additionally, this method necessitates an initial downsampling step to reduce network computations. In the case of high-resolution (4K) images, this downsampling process entails a substantial reduction factor of up to 16 times (typically downsampled to 256 x 256 resolution). Consequently, this results in significant loss of image details and subsequent degradation in enhancement performance.
frameworks, leading to state-
To alleviate the above problems, this work focuses on integrating global and local operators to facilitate comprehensive tone mapping. Drawing inspiration from the reversible Laplacian pyramid decomposition [3] and the classical local tone mapping operators, the local Laplacian filter [29, 1J, we propose an effective end-to-end framework for the HDR image tone mapping task performing global tone manipulation while preserving local edge details. Specifically, we build a lightweight transformer weight predictor on the bottom of the Laplacian pyramid to predict the pixel-level content- -deeedent weight maps. The input HDR image is trilinear interpolated using the basis 3D LUTs and then multiplied with weighted maps to generate a coarse LDR image. To preserve local edge details and reconstruct the image from the Laplacian pyramid faithfully, we propose an image-adaptive learnable local Laplacian filter (LLF) to refine the high-frequency components while minimizing the use of computationally expensive convolution in the high-resolution components for efficiency. Consequently, we progressively construct a compact network to learn the parameter value maps at each level of the Laplacian pyramid and apply them the remapping function of the local Laplacian filter. Moreover, a fast local Laplacian filter [1] is employed to replace the conventional local Laplacian filter [29] for computational efficiency. Extensive experimental results on two benchmark datasets demonstrate that the proposed method performs favorably against state-of-ther the-art methods.
In conclusion, the highlights of this work can be summarized into three points:
(1) We introduce an effective end-to-end framework for HDR image tone mapping. The network
performs both global tone manipulation and local edge details preservation within the same model.
2

---

Ioputt 16 toxt HDR Imogge
Crncs
11.5
Gid
Uparppi:
Concai
LLF
Upaampli
Upaaneal
Ourpat 8-tnt LDB Insuge
Trandomerr
Busk 3D LUT Fusico Block
Weight Predictor
Adaptive Laplacian Pyramid
Refined Laplacian Pyramid
Figure 1: Overview of the framework. Our method first decomposes the input image I into Laplacian pyramid. The low-frequency image How is fed into a lightweight transformer weight predictor and Basis 3D LUTs fusion block to transform into a low-resolution enhanced image how To adaptively refine the high-frequency components, we progressively learn an image-adaptive local Laplacian filter (LLF) based on both high- and low-frequue low-frequecy images. Then, we perform the remapping function of the local Laplcian filter to refine the figh-fre components while preserving the pyramid reconstruction capability. For the level N - 1, we concatenate the component with the edge map of Low to mitigate potential halo artifacts.
(2) We propose anage-adaptive learnable local Laplacian filter for efficient local edge details preservation, demonstrating remarkable effectiveness when integrated with image-adaptive 3D) LUT.
(3) We conduct extensively experiments on two publically available benchmark datasets. Both qualitative and quantitative results demonstrate that the proposed method performs favorably against state-of-the-art methods.
Proposed Method
2.1 Framework Overview
We propose end-to-end framework to manipulate tone while preserving local edge detail in HDR image tone mapping tasks. The pipeline our proposed method illustrated in Fig. 1. Given an input 16-bit HDR image I R h we initially decompose into adaptive Laplacian pyramid, resulting in a collection of high-frequeng components represented by L= [2o, ... 1.. 1]. as well low-frequency image denoted as Iow Here, N represents the number of decomposition levels in the Laplacian pyramid. The adaptive Laplacian pyramid employs a dynamic adjustment of the decomposition levels to match the resolution of the input image. This adaptive process ensures that the low-frequency image Low achieves a proximity of approximately 64 x 64 resolution. The described decomposition process possesses invertibility, allowing the original image be reconstructed by incremental operations. According to Burt and Adelson [3], each pixel in the low-frequency image Now is averaged over adjacent pixels by means of an octave Gaussian filter, which reflects the global characteristics of the input HDR image, including color and illumination attributes. Mcanwhile, other high-free geghffrequency components contain edge-detailed textures of the image.
Motivated by the characteristics mentioned above of the Laplacian pyramid, we propose to manipu- late tone Low while refining the high-frequency components L progressively to preserve local edge details. In addition, we progressively refine the higher-resolution component conditioned on the lower-resolution one. The proposed framework consists of three parts. Firstly, we introduce lightweight transformer block to process Iio and generate content-dependent weight maps. These predicted weight maps are employed to fuse the basis of 3D LUTS. Subsequently, this adapted representation is used to transform Low into How resulting in the desired tone manipulation. Sec- ondly, we construct parameter value maps by leveraging learned model on the concatenation of [N (Iow). up( (edge([(m))))) where up(...) represents bilinear up-sampling operation and edge((..) denotes for the canny edge detector. These parameter value maps are then employed to perform fast local Laplacian filter [1] on the Laplacian layer of level N - 1. This step effectively
3

---

Weight Phalaser
Weight Prodicker
0
3
0
30
11
25.
62
63
52
62
Refasses
STEETTA
ghos
Bacx IDELITT
Weigh
(a)
(b)
Figure 2: Illustration of the basis 3D LUTs fusion strategy. (a) present the multiple pixel mapping relationships of an image pair; (b) is the conventional basis 3D LUTs fusion strategy; (c) is the pixel-level basis 3D LUT's fusion strategy.
refines the high- high-frequency components while considering the local edge detail information. Lastly, propose efficient and progressive upsampling strategy to further enhance the refinement of the remaining Laplacian layers with higher resolutions. Starting from level = N 2 down to I = O, we sequentially upsample the refined components from the previous level and concatenate them with the corresponding Laplacian layer. Subsequently, we employ a lightweight convolution block to perform another fast local Laplacian filter. This iterative process iterates across multiple levels, effectively refining the high-resolution components. We introduce these modules in detail in the following sections.
2.2 Pixel-level Basis 3D LUTs Fusion
According to the inherent properties of the Laplacian pyramid, the low-frequency image contains properties such as color and illumination of the images. Therefore, we employ 3D LUTss to perform tone manipulation on low cowffreuencyy images Low In RGB color space, a 3D LUT defines 3D lattice that consists of N₂ elements, where N₂ is the number of bins in each color channel. Each element
defines pixel-to-pixt mapping function (i, k)), where i,j, k 0, 1, 1, 76
1 TTN
elements' coordinates within 3D lattice and c indicates color channel. Given an input RGB color (( I ((.).k)* (o,,,,)),, where j, k are indexed by the corresponding RGB value, output 0 is derived by the mapping function follows:
O (.j.k) = M MT (15A)) (....))) (i.jA))-
The mapping capabilities of conventional 3D LUTs are inherently constrained to fixed transformations of pixel values. Fig. 2(a) demonstrates this limitation, where the input image has the same pixel values different locations. However, these locations contain different pixel values in the reference image. While the input image is interpolated through a look-up table, the transformed image retains the transformed pixel values these locations. Consequently, the conventional 3D LUT framework fails to accommodate intricate pixel mapping relationships, thus impeding its efficacy in accurately representing such pixel transformations.
Inspired by [36], we propose an effective 3D LUT fusion strategy to address this inherent limitation. The conventional 3D LUT fusion strategy proposed by [38] is shown in Fig. 2(b), which first utilizes the predicted weights to fuse the multiple 3D LUTs into an image-adaptive one and then performs trilinear interpolation to transform images. In contrast, shown in Fig. 2(c), strategy is first to perform trilinear interpolation with each LUT and then fuse the enhanced image with predicted pixel-level weight maps. In this way, our method accurate representation of the complex pixel mapping relationships through the weight values each pixel. The pixel-level mapping function &,,,0, can be described as follows:
enable a relatively more comprehensive and
N-1
O h.w.c (K.ik) = /.w.eo (Tr.)))) I (.....0T (AJ.A)) (i A
Ib
W A.w M
T ( (R (Lad)) (oA)))
(2)
where O hawic
(i.j.)) is the final pixel-level output, whe represents a pixel-level weight map for N 3D LUTs located at (h, w). Note that our proposed strategy involves the utilization of multiple trilinear

---

interpolations, which may impact the computational speed when applied to high-resolution images. However, since our method operates at the resolution of 64 64, the computational overhead is insignificant. More discussions are provided in the supplementary material.
As shown in Fig. 1, given Low with reduced resolution, we feed it into weight predictor output the content -oneent-depeddett weight maps wh.". Since the weight predictor aims to understand the global context, such as the brightness, color, and tone of image, a transformer backbone is more suitable for extracting global information than a CNN backbone. Therefore, we utilize tiny transformer model proposed by [23] as the weight predictor. The whole model contains only 400K parameters when N = 3.
2.3 Image-adaptive Learnable Local Laplacian Filter
Although the pixel-level basis 3D LUTs fu- sion strategy demonstrates stable and efficient enhancement of input images across various scenes, the transformation of pixel values through weight maps alone still falls short of sig- nificantly improving local detail and contrast, To tackle this limitation, one potential solution is to integrate a local enhancement method with 3D LUT. In this regard, drawing inspiration from the intrinsic characteristies of the Laplacian pyra- mid [3], which involves texture separation, vi- sual attribute separation, and reversible recon- struction, the combination of 3D LUT and the local Laplacian filter [29] can offer substantial benefits.
Parumeter Prediction Block
(PPB)
Parameter V Valuz Maps
Input
Local Laplacian Filter Remapping Function r(i)
Refined Companent
Figure 3: Architecture of the proposed image- adaptive learnable local Laplacian filter (LLF).
Local Laplacian filters are edge-aware local tone mapping operators that define the output image by constructing its Laplacian pyramid coefficient by coefficient. The computation of each coefficient is independent of the others. These coefficients are computed by the following remapping functions rii):
r(i) = -1)={(( g sign(i - g) (li g g sign(t (i - ( gl o,) + +o,) ifi>on o,
ifi or
where g is the coefficient of the Gaussian pyramid each level, which acts as a reference value, (z) = =</|æ]| is a function that returns the sign of real number, a is one parameter that controls the amount of detail increase or decrease, / is another parameter that controls the dynamic range compression or expansion, and o, defines the intensity threshold the separates details from edges.
Nevertheless, the conventional approach described in Eq. 3 necessitates manual parameter adjustment for each input image, leading to a cumbersome and labor-intensive process. To overcome this limitation, we propose an image-adaptive learnable local Laplacian filter (LLF) to learn the parameter value maps for the remapping function. The objective function of the learning scheme can be written as follows:
min a C(rl,,))
where a and are the learned parameter value maps of the Laplacian pyramid, C(...) denotes the loss functions, r(1, g) presents the image-adaptive learnable local Laplacian filter (LLF), and are the coefficients of the Laplacian and Gaussian pyramid, respectively, R is the reference image. Note that the parameter о, does not impact the filter's performance; thus, it is fixed at 0.1 in this paper. Furthermore, to enhance computational efficiency, we have employed the fast local Laplacian filter [1] instead of the conventional local Laplacian filter.
As discussed in See. 2.1, we have ly-1 E R x 2x==1 x3 and Tow I 2R 2 x3 To address potential halo artifacts, we initially employ a Canny edge detector with default parameters to extract the edge map of 1/ Subsequently, we upsample Iow and edge using bilinear operations to match the resolution of 1 and concatenate them. The concatenated components are fed into a Parameter Prediction Block (PPB) as depicted in Fig. 3. The outputs of the PPB are utilized for the
5

---

HDR+
(480p)
HDR+ (original) SSIMT
Mcthods
#Params
PSNRT
SSIMT
LPIPSI
AE.
PSNRT
LPIPS.
E
UPE [35]
23.33
0.852
0.150
7.68
21.54
0.723
0.361
9.88
HDRNet [12]
482K
24.15
0.845
0.110
7.15
23.94
0.796
0.266
6.77
CSRNet [14]
37K
23.72
0.864
0.104
6.67
22.54
0.766
0.284
7.55
DeepLPF [26]
1.72M
25.73
0.902
0.073
6,05
N.A.
N.A.
N.A.
N.A.
LUT [38]
592K
23.29
0.855
0.117
7.16
21.78
0.772
0.303
9,45
sLUT [36]
4.52M
26.13
0.901
0.069
5.34
23.98
0.789
0.242
6.85
CLUT [39]
952K
26.05
0.892
0.088
5.57
24.04
0.789
0.245
6.78
Ours
731K
26.62
0.907
0.063
5.31
25.32
0.849
0.149
6.03
Table 1:
Quantitative comparison on HDR+ [13] dataset. "N.A." means that the results are not available due to insufficient memory of the GPU.
remapping function r(i) refine IN-1:
i -0N-
(5)
Subsequently, we adopt a progressive upsampling strategy to match the refined high-frequency component IN-1 with the remaining high-frequency components. This upsampled component is concatenated with Ix-2- As depicted in Fig. 1, the concatenated vector [ up([x-1)] is feed into another LLF. The refinement process continues iteratively, progressively upsampling until lo is obtained. By applying the same operations as described in Eq. 5, all high-frequency components are effectively refined, leading to a set of refined components (o, (,,..., ... [N=1]. Finally, the result image i is reconstructed using the tone mapped image Low with refined components (lo.4, hi...,, [x-1].
2.4 Overall Training Objective
The proposed framework is trained in supervised scenario by optimizing reconstruction loss. To encourage a faithful global and local enhancement, given a set image pairs (I, R), where I[] and R[i] denote pair of 16-bit input HDR and 8-bit reference LDR image, define the reconstruction loss function as follows:
HxW
C1 = Σ
i
Ríd la
How
Row [ 111),
(6)
where i[i] is the output of network with [[i] input, now [i] is the output of 3D LUT with Tow input, Ro [i] is the low-frequency image of reference image R(j..
[+) as
To make the learned 3D LUTs more stable and robust, some regularization terms from [38], including smoothness term L. and monotonicity term L.. are employed. addition to these terms, we employ an LPIPS loss [40] function that assesses solution concerning perceptually relevant characteristics (e.g., g., the structural contents and detailed textures):
f-= = Σ Σ
l1 (1) - ((R)) 11.
H'W
where (b.)).m. denotes the feature map of layer extracted from a pre-trained AlexNet [20]. To summarize, the complete objective of our proposed model is combined as follows:
L1 + XgC, + Am + ApCp:
(8)
where As; Ams and Ap are hyper-parameters to control the balance of loss functions, In our experiment, these parameters are set to A 0.0001, Am 10, X 0.01.

---

MiT--FiveK (480p)
MIT-FFiek (original) LPIPS
Methods
# Params
PSNRT
SSIM
LPIPS
AEI
PSNR1
SSIMT
E
UPE [35]
999K
21.82
0.839
0.136
9.16
20.41
0.789
0.253
10.81
HDRNet [12]
482K
23.31
0.881
0.075
7.73
22.99
0.868
0.122
7.89
CSRNet [14]
37K
25.31
0.909
0.052
6.17
24.23
0.891
0.099
7.10
DeepLPF [26]
1.72M
24.97
0.897
0.061
6.22
N.A.
N.A.
N.A.
N.A.
LUT [38]
592K
25.10
0,902
0.059
6.10
23.27
0,876
0.111
7.39
sLUT [36]
4.52M
24.67
0.896
0.059
6.39
24.27
0.876
0.103
6.59
CLUT [39]
952K
24.94
0.898
0.058
6.71
23.99
0.874
0.106
7.07
Ours
731K
25.53
0.910
0.055
5.64
||
24.52
0.897
0.081
6.34
Table Quantitative comparison on MIT-Adobe FiveK [4] dataset. "N.A." means that the results are not available due to insufficient memory of the GPU.
3 Experiments
3.1 Experimental Setup
Datasets: We evaluate the performance of our network on two challenging benchmark datasets: MIT-Adobe FiveK [4] and HDR+ burst photography [13]. The MIT-Adobe FiveK dataset is widely recognized as a benchmark for evaluating photographic image adjustments. This dataset comprises 5000 raw images, each retouched by five professional photographers. In line with previous studies [38, 36, 39], we utilize the ExpertC images as the reference images and adopt the same data split, with 4500 image pairs allocated for training and 500 image pairs for testing purposes. The HDR+ dataset is a burst photography dataset collected by the Google camera group to research high dynamic range (HDR) and low-light imaging on mobile cameras. We post-process the aligned and merged frames (DNG images) into 16-bit TIF images as the input and adopt the manually tuned JPG images as the corresponding reference images. We conduct experiments on both the 480p resolution and 4K resolution. The aspect ratios of source images are mostly 4:3 or 3:4.
Evaluation metrics: We employ four commonly used metrics quantitatively evaluate the enhance- ment performance on the datasets as mentioned above. The AE metric is defined based on the T.2 distance in the CIELAB color space. The PSNR and SSIM are calcuated by corresponding functions in skimage. metrrcs library and RGB color space, Note that higher PSNR/SSIM and lower LPIPS/ indicate better performance.
To optimize the network, we employ the Adam optimizer [18] for training. The initial values of the optimizer's parameters, 31 and B2, The initial learning rate set to x 10-4, and we use a batch size of during training. In order augment the data, perform horizontal and vertical flips. The training process consists of 200 epochs. The implementation is conducted on the Pytorch [31] framework with Nvidia Tesla V100 32GB GPUs.
Implementation Details:
0.9 and 0.999, respectively.
3.2 Quantitative Comparison Results
our evaluation, we comprehensively compare our proposed network with state-of-the-art learning- based methods for tone mapping in the camera-imaging pipeline. The methods included in the comparison are UPE [35], DeepLPF [26], HDRNet [12], CSRNet [14], 3DLUT [38], spatial- 3DLUT [36], and CLUT-Net [39]. To simplify the notation,
the abbreviations LUT, sLUT, and CLUT to represent 3DLUT, spatial-aware 3DLUT, and CLUT-Net, respectively, in our comparisons. It is important to note that the input images considered in our evaluation are 16-bit uncompressed images in the CIE XYZ color space, while the reference images are 8-bit compressed images the sRGB color space.
Among the considered methods, DeepLPP and CSRNet are pixel-level methods based on ResNet and U-Net backbone, while HDRNet and UPE belong to patch-level methods, and LUT, sLUT, and CLUT are the image-level methods. Our method also falls within the image-level category. These methods are trained using publicly available source codes with recommended configurations, except
7

---

Input
UPE
HDRNet
CSRNet
DeepLPF
LUT
SLUT
CLUT
Ours
Reference
Figure Visual comparison with state-of -at---fftthheeeee methods on test image from the HDR+ dataset [13]. The error maps in the upper left corner facilitates more precise determination of performance differences. Best viewed in color and by zooming in.
for sLUT.. Since the training code and weights of this method have never been released, we reproduce the results according to the description in the released article.
Tab. presents the quantitative comparison results on the HDR+ dataset for two different resolutions. Notably, our method exhibits significant performance advantage over all competing methods both resolutions, as indicated by the values highlighted bold across all metrics. Specifically, our method achieves notable 0.49dB improvement in PSNR compared to the second-best method, sLUT [36], at 480p resolution. This advantage becomes even more pronounced (1 (1.25dB) when operating at the original image resolution, demonstrating the robustness of approach for high-resolution images. Similarly, when evaluated on our second benchmark, the MIT-Adobe FiveK dataset (refer to Tab, 2), method consistently demonstrates clear advantage all competing methods. However, for all methods, the FiveK dataset offers limited improvements compared to the HDR+ dataset, which main reasons. Firstly, some reference images in the FiveK dataset suffer
can be attributed from overexposure or oversaturation, presenting challenges for the enhancement methods. Secondly, inconsistencies exist in the reference images adjusted by the same professional photographers, leading to variations between the training and test sets. More discussions can be found in the supplementary material.
3.3 Qualitative Comparison Results
To evaluate our proposed network intuitively, we visually compare enhanced images on the two benchmarks, shown in Fig. 4 and Fig. 5. Note that the input images 16-bit TIF images, which regular display devices can not directly visualize; thus, we compress the 16-bit images into 8-bit images for visualization. These figures show that proposed network consistently delivers visually appealing results on the MIT-Adobe FiveK and HDR+ datasets. For example, in Fig. 4, our method excels preserving intricate details such as tree branches and grass texture while enhancing brightness. Moreover, our results exhibit superior color fidelity and alignment with the reference image. Similarly, 5,g.5, while other methods suffer from poor saturation in the shaded area of the reflected building, our method accurately reproduces the correct colors, resulting in visually pleasing outcome. These findings highlight the effectiveness and superiority of our method tone mapping tasks. More visual results can be found in the supplementary material. Since the central goal of the tone mapping task is to primarily recalibrate the tone the image while compressing the dynamic range, the visual differences between the results produced by the various state-of ohtee--f-te-art methods are minimal. To intuitively demonstrate the visual differences, we utilize the error maps to facilitate more precise identification of performance differences.

---

Input
UPE
HDRNet
CSRNet
DeepLPF
LUT
SLUT
CLUT
Ours
Reference
Figure 5: Visual comparison with state-of-the-art methods on a test image from the MIT-Adobe FiveK dataset [4]. The error maps in the upper left corner facilitates precise determination of performance differences. Best viewed in color and by zooming in.
Framework Component Transformer
Low-fiequency Image Resolution
Metries
Baseline
Weight Mup
+ Learnable Filter
64 x 64
128 x 128
256 256
PSNR
23.16
24.41
(+1.250)
25.34
(+0.930)
26.62 (+1.280)
26.62
26.69
(+0.070)
26.81
(+0.120)
SSIM
0.842
0.856
(-0.014)
0.868 (+0.012)
0.907 (+0.039)
0.907
0.909
(+0.002)
0.913
(+0.004)
LPIPS]
0.113
0.111
(-0.002)
0.101 (-0.010)
0.063 (-0.038)
0.063
0.061
(-0.002)
0.058
(-0.003)
7.04
6.23
(-0.81)
5.92 (-0.31)
5.31 (-0.61)
5.31
5.29
(-0.02)
5.25
(-0.04)
Table 3: Ablation study of framework component and the selection of pyramid layers. All four metrics are reported.
3.4 Ablation Study
Break-down Ablations. We conduct comprehensive breakdown ablations to evaluate the effects of our proposed framework. We train our framework from scratch using paired data from the HDR+ dataset [13] and evaluate its performance on the HDR+ test set. The quantitative results are presented in Tab. We begin with the baseline method, 3D LUT [38], without utilizing pixel-level weight maps or learnable local Laplacian filters. The results show significant degradation, indicating the insufficiency of 3D LUT. When pixel-level weight maps are introduced, the results improve by an average of 1.25 dB. This evidence highlights the successful implementation of the pixel-level basis 3D LUTs fusion strategy discussed in Sec. 2.2. Next, we replace the regular lightweight CNN backbone with tiny transformer backbone proposed by [23]. which contains less than 400K parameters. After deploying the transformer backbone, the model improved by 0.93dB, suggesting that the transformer backbone is more in line with global tone manipulation and benefits generating more visual pleasure LDR images. Furthermore, when employing the image-adaptive learnable local Laplacian filter, the results exhibit a significant improvement of 1.28 dB. This finding indicates that the adageaad learnable local Laplacian filter facilitates the production of more vibrant results. As be seen from Fig combining local Laplacian filter with 3D LUT achieves good visual quality on both global and local enhancement in this challenging case. These results convincingly demonstrate the superiority of our proposed framework in tone mapping tasks.
Selection the pyramid layers. We validate the influence of the number of Laplacian pyramid layers in this section. Our approach employs adaptive Laplacian pyramid, allowing us manipulate the

---

(a) Bascline
(b) + Weight map
(c) Transformer
(d) Learnable Filter
Figure 6: Visual results of ablation study on framework component. (a) the baseline method 3D LUT. (b) apply the pixel-level weight map. (c) deploy the transformer backbone. (d) is utilizing the learnable local Laplacian filter.
number of layers by altering the resolution of the low-frequency image How As shown in Tab. 3, model performs best on all evaluation metrics when the resolution of How set to 256 256. However, the proposed framework requires more computation. A trade-off between computational load and performance is determined by the number of layers in the Laplacian pyramid. The proposed framework remains robust when the resolution is reduced to alleviate the computational burden. For example, reducing the resolution of Low from 256 x 256 to 64 x 64 only marginally decreases the PSNR of the proposed framework from 26.81 to 26.62. Remarkably, this reduction in resolution leads to significant 30% decrease in computational burden. These results validate that the tone attributes presented in relatively low-dimensional space.
4 Conclusion
This paper proposes an effective end-to-end framework for HDR image tone mapping tasks, com- bining global and local enhancements. The proposed framework utilizes the Laplacian pyramid decomposition technique to handle high-resolution HDR images effectively. This approach signifi- cantly reduces computational complexity while simultaneously ensuring uncompromised enhance- ment performance. Global tone manipulation is performed on the low-frequency image using 3D LUTs. image-adaptive learnable local Laplacian filter is proposed to progressively refine the high-frequency components, preserving local edge details and reconstructing the pyramids. Extensive experimental results on two publically available benchmark datasets show our model performs favorably against state-of-the-art methods for both 480p and 4K resolutions.
Acknowledgements: This work supported by the National Natural Science Foundation of China No.62176097, Hubei Provincial Natural Science Foundation of China No.2022CFA055.
References
[1] Mathieu Aubry, Sylvain Paris, Samuel W Hasinoff, Jan Kautz, and Frédo Durand. Fast local laplacian filters: Theory and applications. ACM Transactions Graphics (TOG), 1315) 2014. 2, 3,
[2] Francesco Banterle, Alessandro Artusi, Kurt Debattista, and Alan Chalmers. Advanced high dynamic range imaging. CRC press, 2017.
[3] Peter J Burt and Edward H Adelson. The laplacian pyramid as compact image code. In Readings in computer vision, pages 67 1-679. Elsevier, 1987. 2, 3,
[4] Vladimir Bychkovsky, Sylvain Paris, Eric Chan, and Frédo Durand. Learning photographic global tonal adjustment with database of input/output image pairs. In CVPR 2011, pages 97-104. IEEE, 2011. 7.9
[5] Yu-Sheng Chen, Yu-Ching Wang, Man-Hsin Kao, and Yung- Yu Chuang. Deep photo enhancer: Unpaired learning for image enhancement from photographs with gans. In Proceedings of the IEÉE Conference Computer Vision and Pattern Recognition, pages 6306-6314, 2018.
[6] Paul Debevec and Simon Gibson. tone mapping algorithm for high contrast images. 13th eurographics workshop rendering: Pisa, Italy. Citeseer, 2002. 2
10

---

[7] Frédéric Drago Karol Myszkowski, Thomas Annen, and Norishige Chiba. Adaptive logarithmic mapping for displaying high contrast scenes, In Computer graphies forum, volume 22, pages
Wiley Online Library, 2003. 2
[8] Frédo Durand and Julie Dorsey. Fast bilateral filtering for the display of high-dynamic-range images. In Proceedings of the 29th annual conference on Computer graphics and interactive techniques, pages 257-266, 2002. 2
[9] Gabriel Eilertsen, Rafal Konrad Mantiuk, and Jonas Unger. A comparative review of tone- mapping algorithms for high dynamic range video. In Computer graphics forum, volume 36, pages 565-592. Wiley Online Library, 2017. 1
[10] Raanan Fattal, Dani Lischinski, and Michael Werman. Gradient domain high dynamic range compression. In Proceedings of the 29th annual conference on Computer graphics and interac- five techniques, pages 249-256, 2002.
[11] James A Ferwerda, Sumanta N Pattanaik, Peter Shirley, and Donald Greenberg. A model of visual adaptation for realistic image synthesis. In Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, pages 249-258, 1996.
[12] Michaël Gharbi, Jiawen Chen, Jonathan T Barron, Samuel W Hasinoff, and Frédo Durand. Deep bilateral learning for real-time image enhancement. ACM Transactions on Graphics (TOG), 36(4):1-12, 2017. 2, 6,7
[13] Samuel W Hasinoff, Dillon Sharlet, Ryan Geiss, Andrew Adams, Jonathan T Barron, Florian Kainz, Jiawen Chen, and Mare Levoy. Burst photography for high dynamic range and low-light imaging on mobile cameras. ACM Transactions on Graphics (ToG), 35(6):1-12, 2016. 6, 7,
[14] Jingwen He, Yihao Liu, Yu Qiao, and Chao Dong. Conditional sequential modulation for efficient global image retouching, In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings. Part XIII 16, pages 679-695. Springer, 2020. 6,7
[15] James Hegarty, John Brunhaver, Zachary Vetioo, Jonathan Ragan-Kellee Noy Cohen, Steven Bell, Artem Vasilyev, Mark Horowitz, and Pat Hanrahan. Darkroom: compiling high-level image processing code into hardware pipelines. ACM Trans. Gruph., 33(4): 2014.
[16] Yuanming Hu, Hao He, Chenxi Xu, Baoyuan Wang, and Stephen Lin. Exposure: A white-box photo post-processing framework. ACM Transactions on Graphics (TOG), 37(2):1-17, 2018.
[17] Jie Huang, Zhiwei Xiong, Xueyang Fu, Dong Liu, and Zheng-Jun Zha. Hybrid image enhance- ment with progressive laplacian enhancing unit. In Proceedings of the 27th ACM International Conference on Multimedia, pages 1614-1622, 2019. 2
[18] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint
arXiv: 2014.7 7
[19] Satoshi Kosugi and Toshihiko Yamasaki. Unpaired image enhancement featuring reinforcement- leamiiggcoonroolled image editing software. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 1196-1112 11303, 2020. 2
[20] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. Communications of the ACM, 60(6):84-90, 2017. 6
[21] Jiangtao Kuang. Garrett M Johnson, and Mark D Fairchild. icam06: refined image appearance model for hdr image rendering. Journal of Visual Communication and Image Representation,
18(5):406-414, 2007. 2
[22] Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. nature, $21 5715553))44364444 2015. 2
[23] Yawei Li, Yuchen Fan, Xiaoyu Xiang, Denis Demandolx, Rakesh Ranjan, Radu Timofte, and Lue Van Gool. Efficient and explicit modelling of image hierarchies for image restoration. arXiv preprint arXiv:2303.00744 2023, 5.
[24] Yuanzhen Li, Lavanya Sharan, and Edward Adelson. Compressing and companding high dynamic range images with subband architectures. ACM transactions on graphics (TOG), 24(3):83-884, 2005.
[25] Rafal Mantiuk, Grzegorz Krawezyk, Dorota Zdrojewska, Radostaw Mantiuk, Karol Myszkowski, and Hans-Peter Seidel. High dynamic range imaging. na, 2015.
11

---

[26]
Sean Moran, Pierre Marza, Steven McDonagh, Sarah Parisot, and Gregory Slabaugh. Deeplpf: Deep local parametric filters for image enhancement. In Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition, pages 22826-12835, 2020. 6,
[27] Ravi Teja Mullapudi, Andrew Adams, Dillon Sharlet, Jonathan Ragan-Kelley, and Kayvon Fatahalian. Automatically scheduling halide image processing pipelines. ACM Transactions on
Graphics (TOG), 35(4):1-11, 2016. 2
[28] A van Oppenheim, Ronald Schafer, and Thomas Stockham. Nonlinear filtering of multiplied and convolved signals. IEEE transactions on audio and electroacousticc, 16(3):437-466, 1968.
[29] Sylvain Paris, Samuel W Hasinoff, and Jan Kautz. Local laplacian filters: Edge-aware image processing with laplacian pyramid. ACM Trans. Graph., 30(4):68, 2011. 2,5
[30] Jongchan Park, Joon- Young Lec. Donggeun Yoo, and In So Kweon. Distort-and-recover Color enhancement using deep reinforcement learning. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5928-5936, 2018. 2
[31] Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary De Vito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. Openreview.net, 2017. 7
[32] Jonathan Ragan- Belan,Kelley, Andrew Adams, Sylvain Paris, Marc Levoy, Saman Amarasinghe, and Frédo Durand. Decoupling algorithms from schedules for easy optimization of image processing pipelines. ACM Transactions on Graphics (TOG), (14): 2012.
[33] Erik Reinhard and Kate Devlin. Dynamic range reduction inspired by photoreceptor physiology. IEEE transactions visualization and computer graphics, 1111):13-2) 13,24,, 2005. 2
[34] Erik Reinhard, Michael Stark, Peter Shirley, and James Ferwerda. Photographic tone reproduc- tion for digital images. In Proceedings of the 29th annual conference on Computer graphics and interactive techniques, pages 267-276, 2002. 1,2
[35] Ruixing Wang, Qing Zhang, Chi-Wing Fu, Xiaoyong Shen, Wei-Shi Zheng, and Jiaya Jia. Underexposed photo enhancement using deep illumination estimation. In Proceedings the IEEE/CVF conference on computer vision and pattern recognition, pages 6849 6857, 2019. 2, 6,7
[36] Tao Wang, Yong Li, Jingyang Peng, Yipeng Ma, Xian Wang, Fenglong Song, and Youliang Yan. Real-time image enhancer via learnable spatial-aware 3d lookup tables. In Proceedings of the IEEE/CVF International Conference Computer Vision, pages 2471-2480, 2021. 2, 4, 6, 7, 8
[37] Zhicheng Yan, Hao Zhang, Baoyuan Wang, Sylvain Paris, and Yizhou Yu. Automatic photo adjustment using deep neural networks. ACM Transactions on Graphics (TOG), 35(2):1-15, 2016. 2
[38] Hui Zeng, Jianrui Cai, Lida Li, Zisheng Cao, and Lei Zhang. Learning image-adaptive 3d lookup tables for high performance photo enhancement in real-time. IEEE Transactions on Puttern Analysis and Machine Intelligence, 44(4):2058-207) 2020. 2, 4, 6,
[39] Fengyi Zhang, Hui Zeng, Tianjun Zhang, and Lin Zhang. Clut-net: Learning adaptively compressed representations of 3dluts for lightweight image enhancement. In Proceedings of the 30th ACM International Conference on Multimedia, pages 6493-6501, 2022. 2, 6,7
[40] Richard Zhang, Phillip Isola, Alexei Efros, Eli Shechtman, and Oliver Wang. The unrea- sonable effectiveness of deep features perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 586-595, 2018. 6
[41] Bolun Zheng, Shanxin Yuan, Gregory Slabaugh, and Ales Leonardis. Image demoireing with learnable bandpass filters. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3636-3645, 2020. 2
12


Reviews:
[{"review_id": "ju7N5PgUCk", "reviewer": "Reviewer_nHJa", "summary": "This paper proposed a pyramid reconstruction network for tone mapping via a combination of existing lookup table and local Laplacian filter techniques, by utilizing closed-form Laplacian pyramid decomposition and reconstruction.\n\nSpecifically, the image-adaptive 3D LUTs is employed to manipulate the tone in the low-frequency image by leveraging the specific characteristics of the frequency information. The local Laplacian filter is utilized to refine the edge details in the high-frequency components in an adaptive manner.\n\nExtensive experimental results on two benchmark datasets demonstrate that the proposed method performs favorably against state-of-the-art methods.\n", "questions": "- How about the performance for the setting with image-adaptive Local Laplacian Filter disabled? This result helps to determine the effectiveness of the proposed technology.\n- The visual comparison in Fig. 4 is not easy to understand. What does the heat map in the upper left corner mean?", "limitations": "Not applicable", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Originality: The idea of optimize existing Laplacian pyramid decomposition and reconstruction based tone mapping method via the combination of lookup table and local Laplacian filter techniques is interesting and reasonable. Besides, the proposed image-adaptive learnable local Laplacian filter can preserve the local edge details.\n- Clarity: This paper is well-organized and easy to follow.\n", "weaknesses": "- The algorithm description is not clear enough：\\textit{e.g.,} the definition of objective function for the image-adaptive learnable local Laplacian filter in Eq. (4) is missing.\n\n- Insufficient experimental analysis：\n  - The runtime analyse are not provided.\n  - Quantitative comparison of different methods on HDR+ datasets is missing.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your valuable comments. Now we respond to your concerns one by one.\n\n(i) Unclear algorithm description.\n\nThe formula of the objective function for the image-adaptive learnable local Laplacian filter is similar to Eq. (3), albeit with the distinction that the parameters within it are learnable by a compact network. We will further refine this aspect in our final version to enhance its accessibility and clarity for readers.\n\n(ii) Runtime analyses.\n\nWe draw on prior literature that only provides the number of parameters. To better characterize our model's computational complexity and efficiency, we provide a more diverse set of metrics, including the number of parameters, the multiply-accumulate operations (MACs), and the runtime. We will include these metrics in the final version of our paper and present them in subsequent tables:\n\n| Methods   | $\\#$Params | MACs      | Runtime   |\n|-----------|------------|-----------|-----------|\n| UPE       | 999K       | 1.146G    | 8.42ms    |\n| HDRNet    | 482K       | 1.103G    | 4.56ms    |\n| CSRNet    | 37K        | 52.843G   | 48.28ms   |\n| DeepLPF   | 1.72M      | 454.422G  | 386.62ms  |\n| LUT       | 592K       | 0.676G    | 2.67ms    |\n| sLUT      | 4.52M      | 20.316G   | 9.63ms    |\n| CLUT      | 952K       | 9.391G    | 8.35ms    |\n| [3]       | 4.45M      | 1410.048G | 162.23ms  |\n| Ours      | 731K       | 2.923G    | 20.51ms   |\n\nAll presented results are obtained using a 32GB NVIDIA V100 GPU. Runtime values are averaged over 1000 images, each with dimensions of $3840\\times2160\\times3$. Similarly, Multiply-Accumulate Operations (MACs) are computed based on the input dimensions $3840\\times2160\\times3$.\n\nThe results show that the extensive computational demands of CSRNet, DeepLPF, and [3] (Reviewer zrx6 requires a comparison with [3]) lead to slow processing. These results can be attributed to the fact that they are pixel-level methods, rendering them more reliant on hardware capabilities, especially when processing large-resolution images. Conversely, our approach makes a trade-off between computational complexity and performance.\n\n(iii) Quantitative comparison of different methods on HDR+ datasets is missing.\n\nIn fact, Table. 2 in our main manuscript provides a quantitative comparative analysis of the HDR+ dataset.\n\n(iv) Experiment about disable the image-adaptive Local Laplacian Filter.\n\nIn fact, as described in Sec. 3.4, we have performed ablation experiments on each component of the model. Table. 3 and Fig. 6 provide the evaluation results of disabling the image-adaptive local Laplace filter qualitatively and quantitatively, respectively.\n\n(v) Why heat map in the upper left.\n\nIt is imperative to clarify that the visual representation in Fig. 4 and Fig. 5 is an error map rather than a heat map. This error map represents the relationship difference derived by subtracting the reference image from the tone-mapped image. The rationale for this presentation lies in the central goal of the tone mapping task - to primarily recalibrate the tone of the image while compressing the dynamic range. Due to these common goals, the visual differences between the results produced by the various state-of-the-art methods are minimal. Therefore, the purpose of utilizing this error map is to facilitate a more precise identification of performance differences. Our final version will add some clarification to avoid any potential misunderstandings.\n\nReference:\n\n[3] Vinker, Yael, Inbar Huberman-Spiegelglas, and Raanan Fattal. \"Unpaired learning for high dynamic range image tone mapping.\" Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021."}, {"review_id": "i684a2Dm5Z", "reviewer": "Reviewer_g4yF", "summary": "The paper proposes an algorithm for the HDR image tone mapping task. It is an end-to-end framework combining global and local enhancements. It uses the Laplacian pyramid decomposition technique. It achieves a balance between computational complexity and performance. Global tone manipulation is performed on low-frequency component. Experiment shows that it out-performs SOTA.", "questions": "see weaknesses", "limitations": "did not talk about potential negative societal impact", "rating": 3, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "It achieves a balance between computational complexity and performance.\nExperiment shows that it out-performs SOTA.\nIt is intuitive to do global tone manipulation on low-frequency image, while using image-adaptive learnable local Laplacian filter for high-frequency component.\nNice ablation studies are performed to quantify the improvements from different modules of the whole algorithm.", "weaknesses": "I wonder why the the transformer weight predictor and the basis 3D LUTs fusion block only exist in the coarsest level of the pyramid. Is there any way to integrate them into all levels?\nIf integrate them into all levels, how much additional computational burden will be introduced?\n\nThe quantitative comparison on HDR+ dataset shows that CSRNet and LUT are both smaller than the proposed algorithm. Is there any chance to compare with them with similar amount of computation? E.g. if the proposed algorithm is reduced to 37K in terms of number of parameters, will it still outperform CSRNet? \n\nLUT will use heavy storage. How to compare the proposed algorithm with SOTA in terms of data storage? Number of parameters alone will not tell the whole picture of the efficiency, what is the real running time comparison?\n\n\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your valuable comments. Now we respond to your concerns one by one.\n\n(i) Integrate transformer model and fusion block to all pyramid levels.\n\nPrimarily, the architectural design of our model focuses on the strategic goal of achieving a minimum number of parameters and computational requirements. This deliberate emphasis on efficiency aims to facilitate the deployment of the model on edge devices. Consequently, our approach is centered on minimizing the number of convolutional layers, effectively streamlining the computational footprint of the model.\n\nSecondarily, the rationale for deploying a lightweight transformer model at the coarsest layer stems from the intrinsic nature of the base of the Laplace pyramid. The coarsest layer of the Laplace pyramid is the low-frequency (global) information of the image, and the transformer model collects the global information of the image better than the CNN model. Furthermore, the placement of the LUT fusion module in proximity to this layer is deliberate. The LUT is a global tone-mapping operator, rendering it adept at processing global information. At the same time, the fast processing capability of the LUT mitigates the slower execution of the transformer model, thus effectively compensating for this temporal trade-off. \n\n(ii) Reduce the number of parameters and compare with CSRNet and LUT.\n\nThe number of parameters in our model is distributed in three different parts. The first part consists of the transformer model, which requires 329K parameters; the second part consists of the LUT, which requires 315K parameters; and the last part is the compact network that predicts the parameters of the Laplace filter, which requires 87K parameters. The transformer model itself is characterized by a large number of parameters, so it becomes a challenge to reduce the number of parameters to 37K. To achieve this goal, we adopted a strategic approach of replacing the transformer model with a conventional CNN model while removing the LUT component and reducing the number of parameters of the compact network. As a result of the concerted effort, a target parameter count of 37K was finally achieved. Subsequently, the generated model was juxtaposed with CSRNet and LUT, and the results for the HDR+ dataset (480p) are fully presented in the following table:\n\n| Methods      | $\\#$Params | PSNR  | SSIM  | LPIPS | $\\triangle$ E |\n|--------------|------------|-------|-------|-------|--------------|\n| CSRNet       | 37K        | 23.72 | 0.864 | 0.104 | 6.67         |\n| Ours (37)    | 37K        | 23.91 | 0.872 | 0.098 | 6.32         |\n| LUT          | 592K       | 23.29 | 0.855 | 0.117 | 7.16         |\n| Ours (592)   | 592K       | 25.53 | 0.891 | 0.077 | 5.84         |\n\n(iii) The data storage of LUT.\n\nThe data storage requirement for a Look-Up Table (LUT) is notably compact. For instance, the LUT employed in our study comprises dimensions of 33x33x33, culminating in a mere storage demand of 3x33x33x33 bits, translating to 107,811 bits, or approximately 0.1 megabytes (MB). Notably, our study incorporates three LUTs, which merge to form a singular LUT. Consequently, our study's cumulative storage allocation for all LUTs is a mere 0.3 MB.\n\n(iv) Runtime analyses.\n\nPlease refer to the response to the question (\\romannumeral2) of reviewer nHJa for more comprehensive results on computational complexity and efficiency.\n\n(v) Limitation and potential negative societal impact.\n\nIn fact, we analyze the broader impacts and limitations of our work in Sec. 4 and Sec. 5 of the supplementary materials. Regrettably, limited by the constraints imposed on paper length, we apologize for placing these pivotal sections in the supplementary materials."}, {"review_id": "qgcTA5LRsM", "reviewer": "Reviewer_1rH8", "summary": "This paper introduces an effective end-to-end framework for HDR image tone mapping. The network performs both global tone manipulation and local edge details preservation within the same model. The proposed method learns an image-adaptive learnable local Laplacian filter for local edge details preservation.", "questions": "1. Why are the key metrics of baselines reported in this paper significantly lower than those reported in their original paper? For example, the PSNR of CLUT[39] on the HDR+ dataset is 26.98dB, while this figure is only 26.05dB in your Table 1, the PSNR of sLUT [36]  on the HDR+ dataset is 26.94dB, while this figure is 26.13dB in your paper, all these figures are much higher than yours. The same issues occur in Table 2.", "limitations": "The limitation is discussed in Supp materials.", "rating": 7, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This paper is well-organized and easy to read.\n2. The experiment results show its effectiveness on two benchmarks, as well as the visual results.\n3. The proposed learnable local Laplacian filter shows remarkable improvement when integrated with image-adaptive 3D LUT.", "weaknesses": "1. Only limited benchmarks are valued, unlike previous work like CLUT, evaluated on both FiveK, HDR+, and PPR10K, so more evaluation results are encouraged.\n2. There is no inference time presented in this work, which is another key information for enhancement applications.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your valuable comments. Now we respond to your concerns one by one.\n\n(i) Evaluation on PPR10K dataset.\n\nThe PPR10K dataset is designed explicitly for portrait photo retouching, which deviates from the domain of tone mapping datasets. Notably, the CLUT being compared in our study is subject to adaptation. In the original paper of CLUT, the input image consisted of a nonlinear image (8-bit) that had been tone-mapped. In contrast, in our study, we reconfigured it to include the HDR radiogram (16-bit) as input. Consequently, since the PPR10K dataset does not contain HDR images, our model was not evaluated on this dataset.\n\nNonetheless, it is essential to emphasize that our model can be readily extended for image enhancement purposes. We intend to explore this aspect by evaluating the PPR10K dataset in future work. We apologize for any ambiguity in this regard and will give a more precise elucidation in the final version.\n\n(ii) Runtime analyses.\n\nPlease refer to the response to the question (ii) of reviewer nHJa for more comprehensive results on computational complexity and efficiency.\n\n(iii) Metrics of baselines lower than original paper.\n\nFirstly, it is essential to acknowledge that our primary task is tone mapping. Consequently, the CLUT and sLUT referenced in our study differ from their original counterparts, mainly in input images. Notably, both CLUT and sLUT considered in our work employ an HDR radiance map (16-bit) as the input image, diverging from their original papers where an already tone-mapped non-linear image (8-bit) is utilized.\n\nSecondly, it is worth mentioning that the source code of sLUT is still not publicly available. Despite our efforts to reproduce its results, we are still trying to match the original paper's results. Notably, the original CLUT paper specifies the integration of CLUT into sLUT, thus adding to the complexity of reproducing the results of the original paper. This combination of factors led to our findings' inconsistency with the original article's results. \n\nMoreover, these results were meticulously reviewed and endorsed by the authors of CLUT before our submission."}, {"review_id": "y7zT4KbxPR", "reviewer": "Reviewer_zrx6", "summary": "The paper introduces a novel framework for converting high dynamic range (HDR) images to low dynamic range (LDR) images, a crucial task in digital imaging. The proposed framework aims to improve the balance between image enhancement and computational efficiency by incorporating global and local tone mapping operators.\nTraditional methods are either global, processing the whole image uniformly but often missing fine details, or local, capturing fine details with higher computational requirements. To overcome this dichotomy, the authors propose integrating 3D LookUp Tables (3D LUT) as a global operator and local Laplacian filters as a local operator into one end-to-end system. The 3D LUTs are used for global tone manipulation, while local Laplacian filters refine high-frequency edge details.\nMoreover, the authors introduce a learnable local Laplacian filter (LLF) and a lightweight network that progressively learns parameter value maps from annotated data for the local Laplacian filters. This approach reduces manual tuning, which is a limitation of existing methods.\nThe results of the extensive experiments on two benchmark datasets demonstrate that the proposed framework outperforms state-of-the-art methods in terms of both image quality and computational efficiency. The paper contributes to digital imaging by offering an efficient solution to HDR image tone mapping while preserving important details.", "questions": "See the section above.", "limitations": "The paper does not discuss the limitation of the proposed method. I believe the proposed method will not have a potential negative societal impact.", "rating": 7, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The paper is well-written and structured. The authors clearly explain the problems with existing methods and why their proposed framework offers a solution. Concepts are clearly defined, and the methodology is explained in a detailed and straightforward manner. Diagrams, equations, and examples effectively support the reader's understanding.", "weaknesses": "- The paper misses the discussions and comparisons of many related methods [1-3]. These methods are all published in good venues within the past five years and thus deserve a fair discussion.\n- Why the evaluation metrics do not contain TMQI, Fidelity, and Naturalness? These three evaluation metrics are common practice in TMO papers (see [1] for example). As tone mapping is a subjective task, using PSNR or SSIM to evaluate does not provide much information.\n- Some of the compared methods, such as CLUT [39], are tacking low-light image enhancement. Why did the authors compare these methods? The input of tone-mapping is an HDR radiance map, while the input to low-light image enhancement is an already tone-mapped non-linear image. The comparison is unfair and could mislead future work.\n- I strongly suggest the authors revisit the goal of their proposed method. Is it a TMO (tone mapping operator) or low-light image enhancement (LLIE)? I really don't see many TMO methods in the comparison section. If that is the case, I would suggest the authors rename the paper title to low-light image enhancement and change all the terms in the paper.\n- If the proposed method indeed is a TMO, then the first step should be LDR to HDR as the input is a low-light image. Then the authors should also discuss the related work [4-6] and design choices for LDR2HDR.\n- While the authors tested their framework on two benchmark datasets, it would have been beneficial to see the system evaluated on more diverse datasets, including real-world photographs, video frames, and even synthetic images. This could have offered a more comprehensive understanding of the method's robustness and generalizability.\n- The paper lacks a complex analysis of the proposed framework. A comparison of the computational complexity and memory usage between the proposed model and existing methods would give a clearer picture of the performance trade-off.\n- The authors have not thoroughly discussed the potential limitations of their proposed method. Understanding what the model struggles with or where it might fail is essential for future improvements and applications.\n\n[1] Liang, Zhetong, et al. \"A hybrid l1-l0 layer decomposition model for tone mapping.\" Proceedings of the IEEE conference on computer vision and pattern recognition. 2018.\n\n[2] Su, Chien-Chuan, et al. \"Explorable tone mapping operators.\" 2020 25th International Conference on Pattern Recognition (ICPR). IEEE, 2021.\n\n[3] Vinker, Yael, Inbar Huberman-Spiegelglas, and Raanan Fattal. \"Unpaired learning for high dynamic range image tone mapping.\" Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021.\n\n[4] Liu, Yu-Lun, et al. \"Single-image HDR reconstruction by learning to reverse the camera pipeline.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020.\n\n[5] Santos, Marcel Santana, Tsang Ing Ren, and Nima Khademi Kalantari. \"Single image HDR reconstruction using a CNN with masked features and perceptual loss.\" arXiv preprint arXiv:2005.07335 (2020).\n\n[6] Chen, Xiangyu, et al. \"Hdrunet: Single image hdr reconstruction with denoising and dequantization.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the critical comments and good suggestions. Your comments help us to improve the quality of the manuscript. Below we respond to the reviewer’s comments and explain the changes we incorporated into the manuscript.\n\n(i) We thank the reviewer for referring us to these three studies [1]-[3]. We made comparisons between our proposed method and the methods in these studies.\n\n[1] and [2] are conventional tone mapping algorithms. On the other hand, [3] and our approach are deep learning-based but differ notably in methodology. [3] employs a pixel-level neural network for HDR image tone mapping. However, its computational complexity at larger image resolutions limits its application to resource-limited edge devices.\n\nIn contrast, our approach combines a 3D Look-Up Table (3D LUT) with a fast local Laplacian filter, granting it a processing speed advantage. Moreover, our method outperforms both [1] and [3] across all metrics. Qualitative comparisons are illustrated in Fig. 1-4 in our provided rebuttal PDF, while quantitative results are presented in the following table:\n\n| Methods | PSNR   | SSIM   | LPIPS  | $\\triangle$ E | TMQI   | Fidelity | Naturalness |\n|---------|--------|--------|--------|--------------|--------|----------|-------------|\n| [1]     | 18.52  | 0.799  | 0.168  | 12.20        | 0.8862 | 0.8975   | 0.5092      |\n| [2]     | N.A.   | N.A.   | N.A.   | N.A.         | N.A.   | N.A.     | N.A.        |\n| [3]     | 16.21  | 0.709  | 0.215  | 17.60        | 0.8893 | 0.9061   | 0.4723      |\n| Ours    | 26.62  | 0.907  | 0.063  | 5.31         | 0.8977 | 0.9089   | 0.5134      |\n\nN.A. represents no public source code for [2] is accessible. Despite efforts to contact the authors for assistance, no response was received within the rebuttal period. Thus, reproducing their work was unfeasible at this stage. However, our commitment to reproducing [2] remains, and we intend to include comparative results in our final version.\n\n(ii) Evaluation metrics do not contain TMQI, Fidelity, and Naturalness.\n\nOur method was mainly developed considering the possible side effect of noise caused by tone mapping. Therefore, our manuscript only included PSNR, SSIM, LPIPS, and $\\triangle$E. We thank the reviewer for suggesting that TMQI, fidelity, and naturalness should also be considered. Due to rebuttal space constraints, we summarize the comparison results of all methods in the global rebuttal comments. It can be observed that our proposed method also produced acceptable results for TMQI, fidelity, and naturalness.\n\n(iii) Issues regarding the method of comparison.\n\nThe reasons why we compare these methods can be summarized as follows:\n\n(a) CLUT was initially designed for photo retouching rather than low-light image enhancement. In our study, we modify CLUT by transforming the original 8-bit tone-mapped image into a 16-bit HDR radiance map and retraining it. This adaptation broadens CLUT's utility beyond traditional use, including tone mapping.\n\n(b) Given our reliance on lookup tables (LUTs), we compare our approach with state-of-the-art (SOTA) methods using LUTs. We have adapted these LUT-based techniques in our study by converting their input images to 16-bit HDR radiance maps. These adjustments were made for a thorough and fair comparative assessment.\n\n(c) Deep learning-based Tone Mapping Operator (TMO) techniques have seen limited exploration, with only a few articles in reputable venues over the past five years. It's also worth noting that some image enhancement methods can be adapted for TMO with minor changes. Therefore, we reference the classical 3D LUT [7] approach and selectively compare it to deep learning-based methods in our study.\n\nFrom my personal standpoint, within the realm of deep learning-based image processing methods, task names align with the input image nature. When dealing with an 8-bit tone-mapped non-linear image, the task is image enhancement (photo retouching). Conversely, a 16-bit HDR radiance map as input defines the task as tone mapping (imaging pipeline enhancement). This versatility in deep learning allows for achieving image enhancement and tone mapping goals.\n\n(iv) The goal of our proposed method.\n\nOur method is unambiguously tailored for tone mapping, and the paper emphasizes that the input image to the model is a 16-bit HDR radiance map. Remarkably, we seamlessly incorporate a 3D LUT with a fast local Laplacian filter, both of which are classical techniques in the tone mapping field.\n\n(v) LDR to HDR reconstruction.\n\nOur model was critically evaluated using the widely recognized HDR+ and MIT-Adobe FiveK datasets, utilizing 16-bit HDR radiance maps as inputs. Given this, the requirement to convert a single Low Dynamic Range (LDR) image into HDR [4]-[6] seems unnecessary in this context.\n\n(vi) Evaluated on more diverse datasets.\n\nTo the best of our knowledge, the HDR+ and MIT-Adobe FiveK datasets are widely used for tone mapping tasks, and they are real-world photographs. Moreover, [1]-[3] were also evaluated on these datasets. Therefore, there need to be more alternative datasets that can be used for evaluation. In future work, we intend to collect a new dataset to expand the range of evaluation options.\n\n(vii) Runtime analyses.\n\nPlease refer to the response to the question (ii) of reviewer nHJa for more comprehensive results on computational complexity and efficiency.\n\n(viii) Limitation and potential negative societal impact.\n\nIn fact, we analyze the broader impacts and limitations of our work in Sec. 4 and Sec. 5 of the supplementary materials. Regrettably, limited by the constraints imposed on paper length, we apologize for placing these pivotal sections in the supplementary materials.\n\n[7] Hui Zeng, et al. Learning image-adaptive 3d lookup tables for high performance photo enhancement in real-time. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020."}, {"review_id": "oJWPUFnrIm", "reviewer": "Reviewer_h55c", "summary": "In this paper, the authors propose an effective end-to-end framework for HDR image tone mapping tasks, combining global and local enhancements. The proposed framework utilizes the Laplacian pyramid decomposition technique to handle high-resolution HDR images effectively. This approach significantly reduces computational complexity while simultaneously ensuring uncompromised enhancement performance. Global tone manipulation is performed on the low-frequency image using 3D LUTs. An image-adaptive learnable local Laplacian filter is proposed to progressively refine the high-frequency components, preserving local edge details and reconstructing the pyramids. Extensive experimental results on two publically available benchmark datasets show that our model performs favorably against state-of-the-art methods for both 480p and 4K resolutions.\nThe major contribution of this work is combination of high frequency lap filter and low frequency global tone mapping for acheiving new SOTA results for tone mapping. ", "questions": "How about mid frequency treatment ?\n", "limitations": "Improvement may be limited to scenes with complex lightings. ", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "New SOTA results for tone mapping using local learnable lap filters for refining high frequency details. ", "weaknesses": "Generalization to other datasets may need retraining. ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your valuable comments. Now we respond to your concerns one by one.\n\n(i) Generalization to other datasets may need retraining.\n\nSince tone mapping is a critical component of the camera imaging process, tailoring a model for each camera (and dataset) makes more sense. The subjective essence intrinsic to tone mapping tasks implies that universally applicable tone transformations may not uniformly meet the preferences of all users.\n\nIn this regard, our future work will include the collection of diverse datasets, each containing different user preferences. Subsequently, the cumulative histogram statistics of these datasets will be integrated into the network as prior information. Subsequent steps include fine-tuning the pre-trained models to facilitate the realization of different tone mapping styles that accommodate the dynamic range of user preferences.\n\n(ii) Mid-frequency treatment.\n\nTo the best of our knowledge, the Laplace pyramid contains only low-frequency and high-frequency information, which has been confirmed in previous studies [1-5] and has become a consistent observation. According to Burt and Adelson [1], each pixel in the low-frequency image is averaged over adjacent pixels by means of an octave Gaussian filter, which reflects the global characteristics of the input HDR image, including color and illumination attributes. Meanwhile, other high-frequency components contain edge-detailed textures of the image. The tone-mapped images can be obtained by directly utilizing the enhanced low and high-frequency information for pyramid reconstruction.\n\n(iii) Limitations.\n\nIn fact, we analyze the broader impacts and limitations of our work in Sec. 4 and Sec. 5 of the supplementary materials. Regrettably, limited by the constraints imposed on paper length, we apologize for placing these pivotal sections in the supplementary materials. We agree with your comment that our method can not guarantee enhanced performance under complex lighting conditions. This is a common problem widely existing in data-driven deep learning methods. To alleviate this issue, we will augment data samples under complex lighting to fine-tune the network in our future work. \n\n[1] Peter J Burt and Edward H Adelson. The laplacian pyramid as a compact image code. In Readings in computer vision, Elsevier, 1987.\n\n[2] Sylvain Paris, Samuel W Hasinoff, and Jan Kautz. Local laplacian filters: Edge-aware image processing with a laplacian pyramid. ACM Trans. Graph, 2011.\n\n[3] Mathieu Aubry, Sylvain Paris, Samuel W Hasinoff, Jan Kautz, and Frédo Durand. Fast local laplacian filters: Theory and applications. ACM Transactions on Graphics (TOG), 2014.\n\n[4] Lai W S, Huang J B, Ahuja N, et al. Deep laplacian pyramid networks for fast and accurate super-resolution. Proceedings of the IEEE conference on computer vision and pattern recognition. 2017.\n\n[5] Liang J, Zeng H, Zhang L. High-resolution photorealistic image translation in real-time: A laplacian pyramid translation network. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021."}]

Author rebuttal:
We provide qualitative comparison results with [1] and [3] (Reviewer zrx6 requires a comparison with [1]-[3]) in Fig. 1-4 in the attached PDF. The comparison results of [2] are not included since its source code is not publicly available. Despite efforts to contact the authors for assistance, no response was received within the rebuttal period. Thus, reproducing their work was unfeasible at this stage. However, our commitment to reproducing [2] remains, and we intend to include comparative results in our final version.

To better demonstrate the superiority of our methods, we also provide the quantitative comparison results of all methods on TMQI, Fidelity, and Naturalness metrics in the following table:

| Methods | PSNR  | SSIM  | LPIPS | Δ E   | TMQI  | Fidelity | Naturalness |
|---------|-------|-------|-------|-------|-------|----------|-------------|
| UPE     | 23.33 | 0.852 | 0.150 | 7.68  | 0.8789| 0.8951   | 0.4213      |
| HDRNet  | 24.15 | 0.845 | 0.110 | 7.15  | 0.8768| 0.8968   | 0.4212      |
| CSRNet  | 23.72 | 0.864 | 0.104 | 6.67  | 0.8880| 0.8908   | 0.4529      |
| DeepLPF | 25.73 | 0.902 | 0.073 | 6.05  | 0.8752| 0.8820    | 0.4419      |
| LUT     | 23.29 | 0.855 | 0.117 | 7.16  | 0.8818| 0.9018   | 0.4119      |
| sLUT    | 26.13 | 0.901 | 0.069 | 5.34  | 0.8854| 0.9017   | 0.4486      |
| CLUT    | 26.05 | 0.892 | 0.088 | 5.57  | 0.8863| 0.9020    | 0.4705      |
| [1]     | 18.52 | 0.799 | 0.168 | 12.20 | 0.8862| 0.8975   | 0.5092      |
| [2]     | N.A.  | N.A.  | N.A.  | N.A.  | N.A.  | N.A.     | N.A.        |
| [3]     | 16.21 | 0.709 | 0.215 | 17.60 | 0.8893| 0.9061   | 0.4723      |
| Ours    | 26.62 | 0.907 | 0.063 | 5.31  | 0.8977| 0.9089   | 0.5134      |

N.A. represents no public source code for [2] is accessible. All these results were evaluated on the HDR+ 480p dataset. The table above shows that TMQI produces similar results for [1], [3], and our approach. However, as shown in Fig. 1-4, visual results show substantial differences, emphasizing the need for a comprehensive assessment combining all metrics to express performance differences.

[1] Liang, Zhetong, et al. "A hybrid l1-l0 layer decomposition model for tone mapping." Proceedings of the IEEE conference on computer vision and pattern recognition. 2018.

[2] Su, Chien-Chuan, et al. "Explorable tone mapping operators." 2020 25th International Conference on Pattern Recognition (ICPR). IEEE, 2021.

[3] Vinker, Yael, Inbar Huberman-Spiegelglas, and Raanan Fattal. "Unpaired learning for high dynamic range image tone mapping." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021.

Meta-review: Using closed-form Laplacian pyramid decomposition and reconstruction, this paper proposes a pyramid reconstruction network for tone mapping by combining existing lookup tables and local Laplacian filter techniques. In the initial reviews, reviewers raised concerns and offered suggestions regarding deeper analysis, the integration of the transformer model and fusion block into all pyramid levels, the number of parameters, evaluation on more benchmarks, and more evaluation metrics. In the rebuttal, further comparisons are made with SOTA methods, more metrics are provided for evaluation, runtime analysis is provided, and detailed descriptions are presented. Additionally, it explains why the PPR10K dataset was not adopted, and how the reported metrics differ from those in the original papers. A majority of the reviews are positive about the paper after the discussion period.
Decision: Accept (poster)