High-Resolution Image Harmonization with Adaptive-Interval Color Transformation
Authors: Quanling Meng
Qinglin Liu
Zonglin Li
Xiangyuan Lan
Shengping Zhang
Liqiang Nie
Venue: NeurIPS 2024 poster (2024)
Keywords: Image Composition
Image Harmonization
Lookup Table
Primary area: machine_vision
TL;DR: This paper proposes an Adaptive-Interval Color Transformation method to perform pixel-wise color transformation and model local non-linearities of the color transformation for high-resolution image harmonization.

Abstract: Existing high-resolution image harmonization methods typically rely on global color adjustments or the upsampling of parameter maps. However, these methods ignore local variations, leading to inharmonious appearances. To address this problem, we propose an Adaptive-Interval Color Transformation method (AICT), which predicts pixel-wise color transformations and adaptively adjusts the sampling interval to model local non-linearities of the color transformation at high resolution. Specifically, a parameter network is first designed to generate multiple position-dependent 3-dimensional lookup tables (3D LUTs), which use the color and position of each pixel to perform pixel-wise color transformations. Then, to enhance local variations adaptively, we separate a color transform into a cascade of sub-transformations using two 3D LUTs to achieve the non-uniform sampling intervals of the color transform. Finally, a global consistent weight learning method is proposed to predict an image-level weight for each color transform, utilizing global information to enhance the overall harmony. Extensive experiments demonstrate that our AICT achieves state-of-the-art performance with a lightweight architecture. The code is available at https://github.com/aipixel/AICT.

High-Resolution Image Harmonization with Adaptive-Interal Color Transformation
Quanling Meng', Qinglin Liu', Zonglin Li', Xiangyuan Lan",
Shengping Zhang Liqiang Nie
School of Computer Science and Technology. Harbin Institute of Technology, China
Peng Cheng Laboratory, China
quanling meng@hit edu. en, qinglin liu@outlook. com, zonglin li@hit..... adu.t.eu.......eeeemmmm cn lanxy@pel ac. en, s s.zhang@hit. en, nieliqiangdgmail. com
Abstract
Existing high-resolution image harmonization methods typically rely on global color adjustments or the upsampling of parameter maps. However, these methods ignore local variations, leading to inharmonious appearances. To address this problem, we propose an Adaptive-Interve Color Transformation method (AICT), which predicts pixel-wise color transformations and adaptively adjusts the sam- pling interval to model local non-linearities of the color transformation at high resolution. Specifically, parameter network is first designed generate multiple position-depenteee 3-dimensional lookup tables (3D LUT's,, which use the color and position of each pixel to perform pixel-wise color transformations. Then, enhance local variations adaptively, we separate a color transform into a cascade of uub-transformations using two 3D LUTs achieve the non-uniform sampling inter- vals of the color transform. Finally, a global consistent weight learning method is proposed predict image-level weight for each color transform, utilizing global information to enhance the overall harmony. Extensive experiments demonstrate that our AICT achieves tate-offthhe-ar performance with a lightweight architecture. The code is available at https: /gi tithub..... com/ http/aapixelAAAAC.
Introduction
Image composition [3, 2, 24, 51] aims to com- bine foreground object with background im- age to create a realistic composite. which holds significant potential across various domains, in- cluding art, entertainment, commerce [3, 43, 52], and data augmentation [11, 30, 26). However, since the foreground and background may be captured under different conditions, directly pasting the foreground onto the background usu- ally results in inconsistent appearance. To address this problem, image harmonization en- deavors adjust the color of the foreground to seamlessly integrate with the background. which plays a pivotal role in image editing.
90.0 GOXELDD)
Transform
Downsenpll
Adagine
Pommeter
Network
Paturottrr
Figure 1: Our method predicts pixel-wise color transformations at low resolution and adaptively the adjusts sampling interval model local non- linearities of the color transform at high resolution.
Traditional image harmonization methods [5, 19, 27, 28, 29, 36, 37] primarily concentrate on aligning the color statistics of the foreground match the background using hand-crafted features. Since
CCoresponding author (s.zhang @ hit.edu.ca).
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

these methods lack consideration for the content of the composite images, they often yield suboptimal results when there are substantial differences appearance between the foreground and background. With the rapid advance of deep learning, learning-base baaringgg-eddddd methods [38, 17, 9, 8, 22, 15, 33, 6, 44, 13] have become dominant and achieved remarkable progress. These methods usually adopt encoder- decoder based structures to learn the dense pixel-to-pixel transformation between composite images and ground-truth images at a low resolution (c.g., 256 x 256 pixels), while real-world applications increasingly demand high-resolution images. Although these methods can process images with any size theoretically, the computational cost required for high-resolution images is extremely expensive.
Recently, several methods [20, 21, 46, 12] have emerged to tackle the challenge of high-resolution image harmonization. To reduce computational costs, these methods usually use a Low-resolution com- posite image to predict transformation parameters for processing the corresponding nigh-resolution composite image instead of directly generating the final image. These methods can be mainly catego- rized into two groups. Harmonizer [20] and NeCtNet [21] focus predicting image-level parameters to perform global color adjustments. However, these adjustments do not contain any semantic and local information, which leads to identical changes for pixels in different regions with the same color value. On the other hand, DCCF [46] and PCT-Net [12] predict low-resolution parameter maps and then directly upsample them to align with high-resolution composite images for pixel-wise color transformation. However, upsampling low-resolution parameter maps may introduce errors, which fails to model local non-linearities of the color transform at high resolution. In summary, these four methods ignore local color transformations across the foreground regions, which are prone to generate inharmony results in local regions.
To address this problem, we propose an Adaptive-Interval Color Transformation method (AICT) for high-resolution image harmoniza- tion. As shown in Figure 1, AICT predicts pa- rameter maps for pixel-wise color transforma- tions, rather than performing global color adjust- ments. Additionally, it adaptively adjusts the sampling interval to model local non-lincarities of the color transformation at high resolution. To implement this complex transformation, formulate the task as an image-based multi- ple curve estimation problem. Specifically, parameter network is first proposed to gener- multiple curves position-dependeen dimensional lookup tables (3D LUTs), which use the color and position of each pixel to per- form pixel-wis color transformations. Then, to enhance local variations adaptively, we propose an adaptive interval learning method, which sep- arates a color transform into a cascade of sub- transformations using two resition-dependens 3D LUT's to achieve the non-uniform sampling intervals of the color transform. Finally, a global consistent weight learning method is proposed to predict an image-level weight for each color transform, utilizing global information to enhance the overall harmony by modeling the influence of background brightness on the foreground. As shown in Figure 2, AICT achieves state-of-the-as performance in foreground-nomme normalized MSE (IMSE) on the full-resolution images of the iHarmony4 dataset [8] while maintaining model size comparable to PCT-Net [12]. Our main contributions are as follows:
340
Harmonizer
300
DCCF
PCT-Net
240
Bettar
Ours (AICT)
200
Smaller
3-
/1
4
5
18 Model Size (Million)
6
17
19
Figure 2: Model size vs. performance (fMSE score) comparison on the full-resolution images of the iHarmony4 dataset [8]. The proposed method achieves state-of-the-an performance while main- taining a lightweight architecture.
We propose an Adaptive-Interval Color Transformation method to perform pixel-wise color transformations and model local non-linearities of the color transformation for high- resolution image harmonization.
We propose adaptive interval learning method to achieve flexible sampling point allocation and a global consistent weight method to utilize global information to enhance the overall harmony.

---

Extensive experiments demonstrate that the proposed method achieves state-of- otate-of-the-ar performance high-resolution image harmonization while maintaining lightweight and simple network architecture.
2 Related Work
2.1 Image Harmonization
To reduce computational costs, existing high-resolution image harmonization methods [20, 21, 46, 12, 7, 40] usually takes a low-resolution image as input to predict transformation parameters for processing the corresponding high-resolution image instead of directly outputting the final image. For example, Harmonizer [20] employs a neural network to regress filter arguments based on low- resolution images, which are then used for several white-box filters to adjust various aspects of the foreground, including brightness, contrast, and other characteristics. S²CRNet [21] focuses on extracting aattial-sepaatted embeddings from low-resolution images to predict parameters of the piece-wise curve mapping for performing color-wiss transformations. Similarly. Wang et al. [40] utilize down-sampled images predict global RGB curves for performing color correction higher resolutions. Furthermore, they propose to predict and upsample low-resolution shading maps to address local tonal variations. These three methods predict image-level parameters perform global color adjustments. Unlike these methods, DCCF [46] processes low-resolution images to acquire human comprehensible neural filter maps, which are subsequently upsampled and applied to the original input image. PCT-Net [12] predicts affine transformation parameter maps based on low-resolution images and upsamples them match high-resolution images. Both methods predict low-resolution parameter maps and then upsample them directly to align with high-resolution images for pixel-wise transformations. In addition, CDTNet [7] performs pixel-to-pin exxx-el transformation at low resolution and color-to co-color transformation at high resolution in parallel. It subsequently utilizes a refinement module to integrate the two intermediate outputs. The above methods typically rely on global color adjustments or the upsampling of parameter maps, which ignore local variations. Our method performs gixel-wise color transformations and models local non-linearities for high-resolution image harmonization.
2.2 LUT-based Image Enhancement
lookup table (LUT) defines a of tables consisting of output values, where each value can be addressed by a set of indices. When provided with input combination, it can output a corresponding value by performing lookup and interpolation operations, so it usually serves as an effective and efficient representation of a univariate or multivariate function. Zeng et al. [49] propose to learn multiple basis 3D L.TTs and predict content-dependent weights to fuse them into an image-adaptive LUT for photo enhancement. To consider global scenarios and local spatial information, Wang et al. [41] introduce lightweight two-head weight predictor for image-level scenario adaptation and pixel-wise category fusion. Yang et al. [47] achieve more flexible sampling point allocation in 3D LUTs by adaptively learning the non-uniform sampling intervals in the 3D color space. To enhance the expressiveness of the LUT, Yang et al. [48] decompose single color transform into cascade of sub-transforms and use ID LUT's to increase cell utilization within the 3D LUT. Zhang et al. [50] employ hash techniques to reduce the space complexity of 3D LUTs. Liu et al. [23] propose to learn a context map for the pixel-level category and group of image-adaptive coefficients for achieving context-aware 4D LUT. These methods typically predict several weight parameters fuse pre-trained LUT's for global RGB-to-RGi RGB transformations, while our method predicts position-dependen. 31) LUTs for pixel-wise color transformations.
3 Method
Given a composite image E Hxx333 and a binary mask M E {0, "Fxw that indicates the foreground region be harmonized, image harmonization aims to adjust the color of the foreground region to obtain harmonized image / R close to the ground truth image To achieve high-resolution image harmonization task as an image-based multiple curve estimation problem and propose an Adaptive-Interval Color Transformation method (AICT) as shown in Figure 3. AICT consists of a high resolution (HR)
Hx x3
low computational cost, we formulate the
3

---

High Resolution Branch
x(L
8(4
2(A,))
Combination
(G.).R((,))
Lookup & Lineer
e.D
Lookup & Linear Interpolation
Interpolation
Reconstruction
Downsample
Parameter Network
M
few
COCV
Treeffonn
decomv
Froddem
Operations
COCP
Wi eight Leaming
[w w.
Intermediate Outputs
Module
Low Resolution Branch
Figure 3: The framework of the proposed method. AICT consists of a high resolution (HR) branch and low resolution (LR) branch. In the LR branch, the composite image and composite mask M downsampled to predict two parameter maps C and F. In the HR branch, C' is used to redistribute the color values of 7 into specific ranges of the following F, which achieves adaptively adjusting the sampling interval of the color transformation. Finally, the new color values and pixel coordinates are mapped to final color values using F.
branch and low resolution (LR) branch. In the LR branch, and M first downsampled obtain a low-resolution composite image Now E H' xW' and foreground mask Mew E {0, 1] x'W" where H' < H and W < W. They then fed into parameter network to predict maps C E R x xQ and F g xw'''''xx xQ and they are regarded as curves in the form of position- dependent 3D LUTs, where Q is equal to three times the number of knot points in a curve. In the HR branch, C' is used process color values and pixel coordinates to redistribute the color values of I into specific ranges of the following F for achieving pixel-wise adaptive adjustment of the sampling interval. F' is used to map the redistributed color values and pixel coordinates to final color values, achieving pixel-wise color transformation. Therefore, the color transformation in our method includes two cascaded pixel-wise sub-transformations.
parameter
3.1 Adaptive Interval Learning
In the LR branch, How and Mew fed into the parameter network to produce C and F. As shown in Figure 3, we use the Transformer encoder [13] to construct the parameter network, which consists of multiple attention layers. Firstly, the low-resolution composite image How and foreground mask Miow divided into patches, which are then projected into the embedding space. Positional encodings are added to the embedded patches, which are processed by the Transformer encoder. The output of the Transformer encoder is reassembled obtain the feature Ti 664x64x256 and deconvolution layer is applied to reduce the feature channel number and perform upsampling to obtain the feature T2 E TR H' XW' x12, Subsequently, Tg is processed through a 1 x convolution to obtain the parameter map C E 22'''x"xxQ We individually process each color channel and divide C' into three parameter maps CR E Rl'''W'x K. CG E "f' XW' x K and C E "''''""XKK where Q is equal to 3 x K. Taking the red channel of an RGB image as an example, the corresponding parameter map is C To achieve adaptive interval learning, we need to reconstruct the parameter map CR, The softmax function is used to obtain the normalized interval VR [0. 1| xW xM Softmax axis = 3). where axis indicates that normalization performed along the third dimension [47] OK We perform cumulative summation in the third dimension of VR and then add an origin for each position to obtain the sampling coordinates KR [0, H|''' x x which can be expressed as R [2; Cumsum (VR, axis = 3)], where Z is H' x W' x 1 matrix filled with zero values, and the : denotes the concatenation operation. In such way, each value in KR is within the range of to and maintains the monotone increasing properties along the third dimension (KR c,b,; < KR e.b.j*
for c, b E 1255 i,j E 1M and i j).

---

We use position-dependee dependent 3D LUTs to achieve pixel-wise adaptive adjustment of the sampling interval. The position-dependent 3D LUT maps spatial coordinates and color values to new color values [35]. We treat K as
deeerdded 3D LUT,, which utilizes lookup and interpolation operations to serve as color mapping curve. For a given coordinate (f,j) and its color value x(1. j) in the red channel of I, we can find the corresponding position (u, U, w)) in K* where u ' - V - J-T. w j) K-1 and Cmax represents the maximum color value in the image. Based on (u, U, w),, we can obtain adjacent 8 sampling points in K and perform trilinear interpolation to produce new color value (ii,j). The entire process can be formulated as
Cuss
{ii,j = t(l(t,j x(..)))
(I)
where and I denote trilinear interpolation and lookup operations, respectively. The upper section of Figure demonstrates the color redistribution process, Here, each coordinate and its corresponding color value are mapped new color value using K Similarly, we can obtain redistributed color values for the other color channels in the same way.
3.2 Pixel-Wiss Color Transform
After obtaining the redistributed color values, we utilize the parameter map F to achieve pixel-wise color trans- formation. As shown in Figure 3, we apply
KR
x(i.)
20.5)
convolution to the fea- ture T2 for obtaining the intermedi- ate parameter map R. E R xW xQ, which are divided into three param- R
Color Redistribution
eter maps RR E R H''''' xK and RB H'''
A(L
$0.D
according to the color channel. When inserting the foreground into a new background, the average brightness of the foreground will be influenced by the background. For example, the inserted foreground may experience overall darkening, brightening, bias toward a certain color. As shown in Figure 5, the foreground region dergoes darkening after image harmonization. Therefore, we propose globally consistent weight learning method and design a weight learning module to learn image-level parameters for controlling overall color transformations. These parameters can also be regarded as scene classification, which are adjusted based on different scenes. The weight learning module consists of two 3x3 convolutional layers, Batch Normalization (BN) layers [18], max-pooling layers, ReLU activation functions, and one fully-connected layer, making it lightweight. It processes Ti to predict weight vectors w R w E IR K and w E RK which are used to multiply with RR RG, and Ra to obtain the final parameter maps f. E I XX'X F. e IR xW'* and f.B e ig x respectively.
Prediction
Figure 4: The illustration of the color redistribution and result prediction. The coordinates and color values are first mapped to new color values using K Then, the final color values obtained by using F according to the coordinates and redistributed color values.
We also treat FR. as position- dependent 3D LUT. For a given coor- dinate (i, and its redistributed color value ((i,)) in the red channel of I, lookup and interpolation operators are performed in & produce the final output color value ji. The bottom part of Figure illustrates the result prediction process. Due to the differ- ence in resolution, each pixel the pa- rameter maps KR and p& corresponds to local region of the composite im- age. However, the color values in
Composte Image
Ground trut
Figure 5: The illustration of the overall impact of the back- ground the foreground. The foreground is outlined in red. The pixel histogram statistics indicate that the overall foreground region undergoes darkening after the process of image harmonization.
5

---

region are usually distributed within certain ranges. Therefore, R aims to redistribute these color values across the entire color value range, thereby improving the utilization of sampling points in FR and enhancing the expressiveness of the curve. By combining these two curves (K and FR), our method achieves the capability of adaptively adjusting the sampling intervals. Finally, the result images corresponding to each channel are obtained by transforming the color values at each position of I, which are then concatenated to produce the harmonized image i. As we process each color channel separately for an RGB image, the parameter network needs to predict six curves.
3.3 Loss Functions
During the training phase, the loss Lnggh is calculated in the HR branch based on the difference between I and I. To improve the performance of AICT for images with small foregrounds, adopt the foreground -orounnidd- MSE loss [33], which is formulated as
H Σ
W
ΣΣ
1i.5 2
high
H Σ
max (Amin:
Mast
where Amin is hyper-parameter to stabilize training. By replacing i with How in the framework, can also obtain the low-resolution harmonized image hiw and the loss Luw- We also apply the foreground-normalizes MSE loss minimize the difference between how and the low-resolution version of the ground truth /ow. Overall, our network can be trained by optimizing the combination of the losses above
L = Lnigh + AL
where À is a hyper-parameter that controls the weight of Liow. Our parameter network is trained in an end-to-end manner.
4 Experiments
4.1
Experimental Setting
4.1.1 Dataset and Evaluation Metries
Following the same settings as previous methods [20, 46, 12], train and evaluate method on the iHarmony4 dataset [8], which consists of four subsets (HAdobe5k, HCOCO, HDay2night, and HFlickr) and includes 73146 samples for image harmonization. Each sample contains composite image, a corresponding foreground mask, and a ground truth image. The HAdobe5k, HCOCO, HDay2night, and HFlickr subsets consist of 2160, 4283, 133, and 828 test images, respectively. These subsets have resolutions ranging from 312 x 230 to 6048 x 4032 pixels. The width and height of images in the HCOCO. HDay2night, and HFlickr subsets are all below 1024 pixels, and only the HAdobe5k subset is composed of images with a width or height larger than 1024 pixels.
We also train and evaluate method on the ecHarmony dataset [25], which is constructed by transferring each image across different illumination conditions to simulate natural illumination variations. This dataset contains 3080 training samples and 1180 test samples, and each sample includes a composite image, corresponding foreground mask, and a ground truth image.
Additionally, we evaluate our method against other methods on a real composite dataset [7], which consists of 100 test samples. Each sample contains only a composite image and a corresponding foreground mask. Note that all datasets used in our experiments publicly available.
For the quantitative performance metrics, we calculate several key indicators, including Mean Square Error (MSE), foreground Mean Square Error (fMSE), Peak Signal-to-Noise Ratio (PSNR), and Structural Similarity Index (SSIM) for each individual image the dataset. Subsequently, we compute the average values for the entire dataset and for each specific subset. While MSE serves as a crucial evaluation metric, tends to be biased towards images that contain larger foreground regions due to the variations in foreground sizes across the dataset. This limitation makes MSE less reliable for a comprehensive assessment of image quality. In contrast, fMSE offers a more balanced and equitable evaluation of overall quality, making more suitable choice for our analysis [12].

---

Table 1: Quantitative comparison on the full-resolution test imgges of the iHarmony4 dataset. The best results are marked in bold, and the second best are underlined.
AdaInt [47]
SepLUT
Harmonizer
DCCF
PCT-Net
Dataset
Metric
Our AICT
[48]
[20]
[46]
[12]
FMSFL
216.12
208.11
196.12
195.54
149.39
133.45
HAdohe5k
MSF.
30.38
26.53
24.37
23.12
19.35
17.09
PSNRT
38.22
38.19
37.80
37.79
39.97
40.32
SSIM
0.9856
0.9844
0.9339
0,9858
0.9878
0.9878
IMSEL
374.89
380.77
374.96
317.43
245.67
240.62
HCOCO
MSEL
21.76
22,43
20.93
16.85
12.45
12.30
PSNRT
37.94
37.83
37.69
38.71
39.85
39.83
SSIM
0.9918
0,9912
0.9858
0,9930
0.9938
0,9936
IMSEL
699.88
702.16
640.74
715.43
700.65
639.00
HDay2night
MSEL
49.40
$2.98
37.28
55.78
46.47
42,90
PSNRT
37.42
37.13
37.15
37.52
37.25
37,50
SSIM
0,9800
0.9782
0.9548
0.9788
0.9818
0.9819
fMSEL
588.40
580.50
479.26
438.49
357.53
334,11
HFlickr
MSEL
86.08
86.69
69.19
64.63
45.79
43.74
PSNRT
32.63
32.56
33.37
33.61
34.87
35.09
SSIM
0.9817
0,9790
0,9714
0,9844
0,9876
0,9877
IMSEL
358.29
358.51
339.23
302.56
238.27
228.43
MSEL
31.96
31.36
27.62
24.72
18.80
17.76
PSNRT
37.42
37.34
37.23
37.85
39.28
39.40
SSIM
0.9887
0.9876
0.9685
0.9897
0,9911
0.9910
4.1.2 Implementation Details
For the iHarmony4 dataset [8]. our network is trained from scratch by us- ing Adam optimizer with 31 = 0.9, 32 0.999. and e The batch size is set to 4 and the model is trained for 100 epochs. We set the learning rate as 5e for the first 50 epochs and linearly decay to zero over the next 50 epochs. For the network design, set 8, Q 24, H1' 256, and W' 256. For the loss function, is set to 0.01, and Amin is set to 1000 and 100 in the HR branch and LR branch, respectively. We resize training im- ages ensure that the length of their sides does not exceed 2048 pixels due to memory limitations. During the testing phase, we perform image harmonization using full-resolution images. Following [7]. also conduct tests the HAdobe5k subset at two different resolutions (1024 x 1024 and 2048 x 2048). For the ccHarmony dataset [25], use the parameters trained the iHarmon4 dataset the initial parameters, and then fine-tune our network on the training set of the ccHarmony dataset. Following [25], the test 256 256. To augment training samples, we crop the composite image according to random bounding box, the width and height of which are not smaller than the halved width and height of the composite image, respectively. The random horizontal flip is also applied to training samples. Our network is implemented based on the PyTorch framework and trained over approximately 75 hours on a computer equipped with two EPYC 7513 CPUs, 256GB of memory, and two 3090 GPUs.
Table Quantitative comparison on the HAdobe5k subset at 2048 2048 resolution. The best results are marked in bold, and the second best are underlined.
Method
IMSEJ
MSET
PSNRT
SSIMT
iS*AN [33]
271.59
46.37
36.57
0.9838
AdaInt [47]
221.90
31.09
38.10
0.9847
SepLUT [48]
216.15
27.03
37.98
0.9834
CDTNet-512 [7]
159.13
23.35
38.45
0.9853
Harmonizer [20]
208.93
27.79
36.58
0.8992
DCCF [46]
197.23
23.00
38.34
0.9851
PCT-Net [12]
156.56
20.38
39.83
0.9870
Our AICT
147.99
17.92
40.07
0.9871
image size in this dataset is
4.2 Comparison with the State-of-the-arts
We conduct experiments and comparisons on the full-resolution test images of the iHarmony4 dataset [8]. AdaInt [47] and SepLUT [48], originally designed for image enhancement, modified
7

---

and trained from scratch on the iHarmony4 dataset for the image harmonization task. Our AICT has three key differences compared to these methods. Firstly, they predict several weight parameters to fuse pre-trained LUTS,, while our method dynamically predicts entire LUT's based on input. Secondly, they focus on global RGB-to-RGB transformations, whereas our AICT enables pixel-wise color transformations, Lastly, instead of applying adaptive interval learning and separable lookup tables for global sampling adjustments, our AICT achieves pixel-wise sampling interval adjustment to model local non-linearities in color transformation. To the best of our knowledge, only Harmonizer [20], DCCF [46], and PCT-Net [12] perform image harmonization on full-resolution images. Since Xu et al. [46] do not provide fMSE and SSIM values, evaluate DCCF by running the provided code for comparison. Guerreiro et al. [12] propose two types of architectures: a CNN-based encoder-decodee network and network based on Visual Transformer (ViT) [10]. We choose the ViT-based network for comparison as it demonstrates better performance. The quantitative comparison results are shown in Table 1. We observe that AICT outperforms other methods most metrics. Due to the limited amount of training samples in the HDay2night subset, our method achieves lower performance on this subset. To evaluate performance on oowressouution images, we also report quantitative results on the iHarmony4 dataset at a 256 x 256 resolution (see the Appendix). Additionally, we evaluate the harmonization performance on the HAdobe5k subset at 2048 2048 resolution. As shown in Table 2, our method achieves the best performance across all metrics. Furthermore, we compare our method with others on this subset 1024 1024 resolution (see the Appendix).
As shown in Figure 6, we present the qualitative results of AICT against state-of oftec----hhharrr methods on the full-resolution test images of the iHarmony4 dataset [8]. Compared to other methods, the images generated by AICT are closer to the ground truth images, making the composite images more realistic. More qualitative results quantitative comparison results on the real composite dataset [25] to demonstrate the superiority and generalizability of our method (see the Appendix).
presented in the Appendix. Additionally, also present the
To further demonstrate the effectiveness of our method, we also present quantitative comparison results on the ccHarmony dataset [25] at 256 x 256 resolution. As shown in Table 3, our method also achieves the best performance across all metrics.
4.3 Ablation Study
We study the influence of the adaptive in- terval learning method on image harmo- nization performance using the iHarmony4 dataset [8], as shown in Table 4. We first remove the weight learning module and the adaptive interval learning method in AICT, denoted as "Siiggl". Compared to AICT, this method achieves lower performance. We then incorporate the channel-crossine strategy [34, 35] into "Single", denoted as "Cross". The parameter model uses RGB image to predict parameter maps, which in- herently include information from all color channels. Thus, the channel-crossing strat- egy introduces redundancy without improv- ing performance. We also remove the adap-
Table 3: Quantitative comparison on the ccHarmony
dataset at a 256 256 resolution. The best results marked in bold, and the second best are underlined.
Method
IMSEL
MSE,
PSNR
DoveNet [8]
880.94
110.84
31.64
RainNet [22]
519.32
58.11
34.78
IIH [15]
636.28
83.72
33.64
D-HT [14]
514.47
55.73
35.07
is AM [33]
264.84
28.83
36.05
CDTNet [7]
264.51
27.87
36.62
Harmonizer [20]
402.09
43.31
34.68
DCCF [46]
259.83
29.25
36.62
GiftNet [25]
235.20
24.55
37.59
Our AICT
232.66
24.14
38.46
tive interval learning method in AICT, denoted as "w/o Int". Compared to AICT, the performance of this method decreases due to the reduced expressiveness of the curves. To investigate whether using more LUTs can improve performance, we first cascade 2 and LUTs for adaptive interval learning, denoted as "Int x 2" and "Int x 3", respectively. Then, we cascade 2 and 3 LUTss for color transformation, denoted as "Tra 2" and "Tra x 3", respectively. Finally, we cascade 4 and 6 LUTSs for alternating adaptive interval learning and color transformation, denoted as "Alt x 2" and "Alt x 3", respectively. The experimental results show that increasing the number of LUTs for adaptive interval learning does not improve performance. Furthermore, using more LUTSs for color transformation decreases performance due to increased training difficulty. Additionally, we combine AdaInt [47] and ScpLUT [48] for global RGB-to-Red transformations, denoted as "AdaInt
8

---

Composite image
(b) Hermonizer
DCCF
PCT-Net
Ground truth
Figure 6: Qualitative comparison results and етrоr maps. We visualize the error between the harmonized images and the ground truth images. The error maps are normalized for display, and the foreground is outlined in red.
SepLUT". Compared AICT, one for the performance decline of "AdaInt SepLUT" is the neglect of local context.
We conduct ablation studies the global consistent weight learning method, as shown in Table 5. First, remove the weight vector corresponding to the chan- nel, denoted "w/o R". Next, remove the weight vectors for both the and channels, denoted as "w/o RG". Finally, we remove the entire weight learning module, denoted as "w/o Weight". As the number of removed weight vectors increases, the har- monization performance decreases, high- lighting the importance of the global con- sistent weight learning method. We also predict spatially varying weight parameters to weight each sampling point of the pa- rameter map R. denoted as "Spatial". The results for "Spatial" indicate that learning spatially varying weight parameters does not improve performance, the LUT is responsible for pixel-wise color transformations. These observations demonstrate that the proposed adaptive interval learning method and global consistent weight learning method are effective.
Table 4: Ablation studies the adaptive interval learn- ing method. The best results are marked in bold.
Method
IMSEL
MSEJ
PSNR
Single Cross
248.51
19.71
39.15
264.33
21.40
38,90
w/o Int
239.34
18.96
39.20
Int x 2
241.79
19.24
39.17
Int x 3
231.91
18.04
39.38
Tra x 2
456.98
33.94
36.16
Tra x 3
487.60
36.03
35.91
Alt
441.27
33.09
36.28
Alt x 3
489.02
35.95
35.94
AdaInt SepLUT
357.61
31.11
37.47
Our AICT
228.43
17.76
39.40

---

4.4 Hyper-parammter Analyses
We conduct studies on key parameters using the iHarmony4 dataset [8]. includ- ing the number of knots K, the coeffi- cient of the LR branch loss X, and the hyper-parameter Amin in the foreground- normalized MSE loss, as shown in Table 6. We set K to 6, 8, and 10 to analyze the influence of curves with different numbers of knots for the image harmonization task, where K set to in AICT. By comparing "K=6" with AICT, observe that increas- ing the number of knots reduces IMSE and MSE while improving PSNR, indicating that a larger number of knots enhances the harmonization ability of the curves. However, as the number of knots increases, the performance of "K=10" de- creases, suggesting that excessive number of knots increases network parameters. This complicates the training process of the network, making it more challenging to achieve optimal performance. To investigate the effect of the LR branch loss, set À to O, 0.01, and 0. where À 0.01 in AICT. When À is set to 0.01, AICT achieves the best performance on all metrics, demonstrating that an optimal weight for the LR branch loss is crucial for the resolutin image harmonization. For the normmazzedd MSE loss, we set Amin to H x W and H' x W' in the HR and LR branch, respectively, which means these objective functions are equivalent MSE functions, denoted as "MSE". The results indicate that using the foreground-norma performance, as it prevents training samples with foregrounds of different sizes from being trained with varying loss magnitudes, ensuring effective training for small foreground images.
Table 5: Ablation studies on the global consistent weight learning method. The best results are marked in bold.
Method
IMSEJ
MSEJ
PSNR
w/o R
230.83
18.25
39.39
w/o RG
232.20
19.10
39.18
w/o Weight
247.48
19.40
39.16
Spatial
230.24
18.03
39.23
Our AICT
228.43
17.76
39.40
MSE loss improves barmonization
5 Conclusion
In this paper, we formulate image har- monization as an image-based multiple curve estimation problem and propose an Adaptive-Interval Color Transforma- tion method, which predicts pixel-wise color transformation and adaptively adjusts the sampling interval to model local non- linearities of the color transformation at high resolution. Specifically, a parameter network is first designed to generate multi- 3D LUTs,
Table 6: Syyper-paramett analyses on the number of knots K, the coefficient of the LR branch loss X, and the hyper-parameter Amin in the foreground-normalized MSE loss. The best results are marked in bold.
Method
fMSEL
MSEJ
PSNRT
K 6
241.67
18.64
39.20
K = 10
245.43
19.69
39.15
X 0
272.56
21.24
38.70
À = 0.1
253.34
20.39
39.02
ple curves as which use the color and position of each pixel to perform pixel-wise color transfor- mation. Then, we separate color trans- form into a cascade of sub-transsormmation using two position-dependeen transform. Finally, a global consistent weight learning method is proposed to predict an image-level weight for each color transform, utilizing global information to enhance the overall harmony. Experi- mental results demonstrate that our method achieves fatte-of-the-nnd performance in high-resolution image harmonization with a lightweight architecture.
MSE
287.04
23.17
38.41
Our AICT
228.43
17.76
39.40
3D LUTs to achieve the non-uniform sampling intervals of the color
Acknowledgements
This work was supported in part by the National Natural Science Foundation of China under Grants 62272134, 62236003 and 62072141, in part by the National Science and Technology Major Project under Grant 2021ZD01 10901, in part by Shenzhen Colleges and Universities Stable Support Program under Grant SKWW2202208111444280000 Province under Grant ZR2024QF065.
and in part by the Natural Science Foundation of Shandong
10

---

References
[1] Ivan Anokhin, Pavel Solovev, Denis Korzhenkov, Alexey Kharlamov, Taras Khakhulin, Aleksei Silvestrov, Sergey Nikolenko, Victor Lempitsky, and Gleb Sterkin. High-rcsolution daytime translation without domain labels. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7488-7497, 2020. 14
[2] Samanch Azadi, Deepak Pathak, Sayna Ebrahimi, and Trevor Darrell. Compositional GAN: learning image-conditional hinary composition. International Journal of Computer Vision, 128:2570-2585, 2020.
[3] Bor-Chun Chen and Andrew Kae. Toward realistic image compositing with adversarial learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8415-8424, 2019.
[4] Qifeng Chen and Vladlen Koltun. Photographic image synthesis with cascaded refinement networks. In P Proceedings of the IEEE International Conference Computer Vision, pages 1-11-, 2017. 14
[5] Daniel Cohen-Or, Or, Olga Sorkine, Ran Gal, Tommer Leyvand, and Ying-Qing Xu. Color harmonization.
ACM Transactions Graphics, 25(3):224-600 2006. 1
[6] Wenyan Cong, Li Niu, Jianfu Zhang, Jing Liang, and Liqing Zhang. Bargainnet: Background-guide domain translation for image harmonization. In IEEE International Conference on Multimedia and Expo, pages 1 6, 2021. 2, 14
[7] Wenyan Cong. Xinhao Tao, Li Niu. Jing Liang, Xuesong Gao, Qihao Sun, and Liqing Zhang. High- resolution image harmonization via collaborative dual transformations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18470-18479, 2022. 3, 6, 7, 8, 14, 15
[8] Wenyan Cong, Jianfu Zhang, Li Niu. Liu Liu, Zhixin Ling, Weiyuan Li, and Liqing Zhang. Dovenet: Deep image harmonization via domain verification. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 8394-8403, 2020. 2, 6, 7, 8, 10, 14, 15
[9] Xiaodong Cun and Chi-Man Pun. Improving the harmony of the composite image by spatial-separated attention module. IEEE Transactions on Image Procexsing, 29:4759-4771, 2020. 2, 14
[10] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenbom, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold. Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint vrXiv:2010.11921 2020. 8
[11] Dehidatta Dwihedi, Ishan Misra, and Martial Hebert. Cut, paste and learn: Surprisingly easy synthesis for instance detection. In Proceedings of the IEEE International Conference on Computer Vision, pages 1301-1310, 2017.
[12] Julian Jorge Andrade Guerreiro, Mitsuru Nakazawa, and Bjöm Stenger. Pet-net: Full resolution image harmonization using pixel-wise color transformations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5917 1927- 2023. 2, 3, 6, 7, 8, 14, 15, 16
[13] Zonghui Guo, Zhaorui Gu, Bing Zheng, Junyu Dong, and Haiyong Zheng. Transformer for image harmonization and beyond. IEEE Transactions Pattern Analysis and Machine Intelligence, 45(1))11290- 12977, 2023. 2, 4, 14
[14] Zonghui Guo, Dongsheng Guo, Haiyong Zheng, Zhaorui Gu, Bing Zheng, and Junyu Dong. Image harmonization with transformer. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pages 11870-14879, 2021. 8, 14
[15] Zonghui Guo, Haiyong Zheng, Yufeng Jiang, Zhaorui Gu, and Bing Zheng. Intrinsic image harmonization. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 16367-
16376, 2021. 2, 8. 14
[16] Yucheng Hang. Bin Xia. Wenming Yang, and Qingmin Liao. SCS-Co: Self-consistent style contrastive learning for image harmonization. In Proceedings of the IEEESCVE Conference on Computer Vision and Pattern Recognition, pages 19678 -9687, 2022. 14
[17] Guoqing Hao. Satoshi lizuka, and Kazuhiro Fukui. Image harmonization with attention- based deep feature modulation. In Proceedings of the British Machine Vision Conference, volume 1, page 2, 2020. 2
[18] Sergey loffe and Christian Szegedy. Batch Normalization: Accelerating deep network training by reducing internal covariate shift. In Proceedings of the International Conference on Machine Learning, volume 37, pages 448 456, 2015. 5

---

[19] Jiaya Jia, Jian Sun, Chi-Keung Tang, and Heung- Yeung Shum. Drag-and-drop pasting, ACM Transactions Graphics, 2563) 2006.
[20] Zhanghan Ke, Chunyi Sun, Lei Zhu, Ke Xu, and Rynson WH Lau. Harmonizer: Learning to perform white-box image and video harmonization. In Proceedings of the European Conference on Computer Vision, pages 690-706, 2022. 2, 3, 6, 7, 8, 14, 15, 16
[21] Jingtang Liang, Xiaodong Cun, Chi-Man Pun, and Jue Wang. Spatial-separated curve rendering network for efficient and high-resolution image harmonization. In Proceedings of the European Conference on Computer Vision, pages 334-349, 2022. 2,
[22] Jun Ling, Han Xue, Li Song. Rong Xie, and Xiao Gu. Region Region-aware aware adaptive instance normalization for image harmonization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9361-999 9361-9370, 2021. 2, 8, 14
[23] Chengxu Liu, Huan Yang. Jianlong Fu, and Xueming Qian. 4d lut: learnable context-aware 4d lookup table for image enhancement. IEEE Transactions on Image Processing, 32:4742 2023. 3
[24] Li Niu, Wenyan Cong, Liu Liu, Yan Hong, Bo Zhang, Jing Liang, and Liqing Zhang. Making images real again: A comprehensive survey on deep image composition. arXiv preprint arXiv:2106.14 14490, 2021. 1
[25] Li Niu, Linfeng Tan, Xinhao Tao, Junyan Cao, Fengjun Guo, Teng I Long, and Liqing Zhang. Deep image harmonization with globally guided feature transformation and relation distillation. In Proceedings of the IEEE/CVF International Conference Computer Vision, pages 7723-7732, 2023. 6, 7, 8, 14
[26] Xi Ouyang, Yu Cheng, Yifan Jiang, Chun-Liang Li, and Pan Zhou. Pedestrian-symthesis gan: Generating pedestrian data in real scene and beyond. arXiv preprint XXXv::884.0224/ 2018.
[27] Patrick Pérez, Michel Gangnet, and Andrew Blake. Poisson image editing. ACM Transactions on Graphics. 22(():313-3188 2003.
[28] Francois Pitie, Anil C Kokaram, and Rozenn Dahyot. N-dimensional probability density function trans- ferand its application to colour transfer. In Proceedings of the IEEE International Conference on Computer Vision, pages 1434-1439, 2005.
[29] Erik Reinhard, Michael Adhikhmin, Bruce Gooch, and Peter Shirley. Color transfer between images. IEEE Computer Graphics and Applications, 21(5):34-41, 2001.
[30] Tal Remez, Jonathan Huang, and Matthew Brown. Learning to segment via cut-and-paste In Proceedings of the European Conference on Computer Vision, pages 37-52, 2018. I
[31] Xuqian Ren and Yifan Liu. Semantic-guided multi-mask image harmonization. In Proceedings of the European Conference Computer Vision, volume 13697. pages 564 579, 2022. 14
[32] Xintian Shen, Jiangning Zhang, Jun Chen, Shipeng Bai, Yue Han, Yabiao Wang, Chengjie Wang, and Yong Liu. Learning global-aware kernel image harmonization. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7535-7544, 2023. 14
[33] Konstantin Sofiuuk, Polina Popenova, and Anton Konushin. Foreground-aware semantic representations for image harmonization. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 1620-1629, 2021. 2, 6, 7, 14
[34] Yuda Song, Hui Qian, and Xin Du. Starenhancer: Learning real-time and style-aware image enhancement.
In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4126-4135, 2021.
[35] Yuda Song, Hui Qian, and Xin Du. Multi-curve translator for high-resolution photorealistic image translation. In European Conference on Computer Vision, pages 126 143. Springer, 2022. 5, 8.8
[36] Kalyan Sunkavalli, Micah K Johnson, Wojeiech Matusik, and Hanspeter Pfister. Multi-scale image harmonization. ACM Transactions on Graphics, 29-4):1-10, 2010.
[37] Michacl W Tao, Micah K Johnson, and Sylvain Paris. Error-tolerant image compositing. International Journal of Computer Vision, 103(2):178-18) 189, 2013. 1
[38] Yi-Hsuan Tsai, Xiaohui Shen, Zhe Lin, Kalyan Sunkavalli, Xin Lu, and Ming Hsuun Yang. Deep image harmonization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 77889-79777 2017. 2
12

---

[39] Jeya Maria Jose Valanarasu. He Zhang. Jianming Zhang, Yilin Wang, Zhe Lin, Jose Echevarria, Yinglan Ma, Zijun Wei, Kalyan Sunkavalli. and Vishal M Patel. Interactive portrait harmonization. International Conference on Learning Representations. 2023. 16
[40] Ke Wang, Michael Gharbi, He Zhang. Zhihao Xia, and Eli Shechtman. Semi-supervised parametric real-world image harmonization. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5927-5936, 2023.
[41] Tao Wang, Yong Li, Jingyang Peng, Yipeng Ma, Xian Wang, Fenglong Song, and Youliang Yan. Real time image enhancer via learnable spatial-aware 3d lookup tables. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2471-2480, 2021. 3
[42] Ting-Chur Ban-Chun Wang, Ming-Yo Yu Liu, Jun-Yan Zhu, Andrew Tao, Jan Kautz, and Bryan Catanzaro. High- resolution image synthesis and semantic manipulation with conditional gans. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pages 8798-8807, 2018. 14
[43] Shuchen Weng, Wenbo Li, Dawei Li. Hongxia Jin, and Boxin Shi. MISC: multi-conditiin injection and apatially-adaptive compositing for conditional person image synthesis. In Proceedings of the IEFF/CVF Conference on Computer Vision and Pattern Recognition, pages 7741-7749, 2020.
[44] Yazhou Xing, Yu Li, Xintao Wang. Ye Zhu, and Qifeng Chen. Composite photograph harmonization with complete background cues. In Proceedings of the ACM International Conference on Multimedia, pages 2296-2304, 2022.
[45] Ke Xu, Gerhard Petrus Hancke, and Rynson WH Lau. Learning image harmonization in the lincar color space. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pages 2570-12579, 2023. 14
[46] Ben Xue, Shenghui Ran, Quan Chen, Rongfei Jia, Binqiang Zhao, and Xing Tang. DCCF: deep compre- hensible color filter learning framework for high-resolution image harmonization. In Proceedings of the European Conference on Computer Vision, pages 300-316. Springer, 2022. 2, 6, 7, 8, 14, 15, 16
[47] Canqian Yang, Meiguang Jin, Xu Jia, Yi Xu, and Ying Chen. Adaint: Learning adaptive intervals for 3d lookup tables on real-time image enhancement. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17522-17531, 2022.
7, 8, 14, 15
[48] Canqian Yang, Meiguang Jin, Yi Xu, Rui Zhang, Ying Chen, and Huaida Liu. Seplut: Separable image- adaptive lookup tables for real-time image enhancement. In Proceedings the European Conference on Computer Vision, volume 13678, pages 201-217, 2022. 3, 7,8, 14, 15
[49] Hui Zeng, Jianrui Cai, Lida Li, Zisheng Cao, and Lei Zhang. Leamming image-adaptive 3d lookup tables for high performance photo enhancement in real-time. IEEE Transuctions on Pattern Analysis and Machine Intelligence, ((4))225582277) 2020. 3
[50] Fengyi Zhang, Lin Zhang, Tianjun Zhang, and Dongqing Wang. Adaptively hashing 3dluts for lightweight real-time image enhancement. In JEEE International Conference on Multimedia and Expo, pages 2771- 2776. IEEE, 2023. 3
[51] Shengping Zhang, Quanling Meng, Qinglin Liu, Liqiang Nie, Bineng Zhong, Xiaopeng Fan, and Rongrong Ji. Interactive object placement with reinforcement learning. In Proceedings of the 40th International Conference on Machine Learning, pages 41510 41522, 2023.
[52] Sang-Hai Zhang, Zheng-Ping Zhou, Bin Liu, Xi Dong, and Peter Hall. What and where: A context-based recommendation system for object insertion. Computational Visual Media, 6 6(1):79-93, 2020. I
13

---

A Appendix
A.1 More Quantitative Results
Table 7: Quantitative comparison on the iHarmony4 dataset at a 256 x 256 resolution. The best
results are marked in bold, and the second best are underlined.
Mathol
IMSEL
HAdabe5k MSEL
PSNR
IMSELL
BCOCO MSEL
PSNR1
IMSEL
2right MSSE)
PSNR1
[MSEL
Hflickr MSEL
PSNR1
[MSE]
All MSEL
PSNR1
DoveNer [8]
340.30
52.32
34.34
531,01
36.72
25.83
1075.71
54.05
35.18
827.03
133.14
30.21
549. 96
52.36
24.75
BaogatiNNet [6]
279,66
39.93
35.34
397.83
24.84
37.03
$35.63
50.99
35.67
655.
97.32
31.34
23
37.82
25.
1E [15]
254.21
45.02
35.20
416.38
24.92
37.16
797.04
55.53
35.96
716.60
105.13
31.34
400:29
38.71
25,90
LET [14]
265.1 11
38.55
36.88
295.30
16.89
38.00
708.42
53.01
37.10
515.45
74.51
33.13
530.78
30.30
37.55
D [13]
242.56
36.83
37.17
274.66
37.10
36.55
736.38
19.68
36.68
471.05
67.58
33.55
56
27.89
37.94
ISAM [33]
173.96
21:88
38.08
266. 19
36.48
39.16
590199
40.59
37.72
443.65
69.67
33.56
264.96
24.44
38.19
DIHHHRNN [18)
NUA
21 16
97.35
N/A
14.01
19.5 64
N/A
50.61
37.68
NIA
60.41
34.03
252.00
22.00
18.
CIYTN= [7]
NIA
20
38.24
NIA
16.25
39.1
N/A
36.72
$7.95
NIA
68.61
33.55
252.00
23.75
38.23
S2CRNGt (21)
NIA
34.91
36.42
NIA
23,22
38.48
N/A
51.67
36.51
NIA
98.73
32.3%
N/A
35.58
37.18
Harmotieer (20)
170.05
21:89
37.64
298.42 42
37.34
38.77
542,07
33.84
37.56
454,06
64.51
33.63
250.51
24.36
37.84
DCCF (46)
172.49
23.45
37.18
272.10 10
14,87
39.52
655.46
45.09
38.08
411.56
61.42
33.84
265
22.05
38.90
Sy-MHH [S1]
SUA
22.04
38.64
N/A
13:05
19. 14
N/A
43.57
36.96
NIA
59.09
34.00
N/A
21.50
18. 18.58
SCS-Ca(16)
165-48
21
38.20
245.54
13.58
30.5 88
406.80
41.75
37.83
903.72
55.83
34:22
258. 95
21:33
38.75
PCTENN-(
157.24
21:25
39.97
208.26 26
10.72
40.78
654.1
44.74
37,65
341.10
44.36
35.13
236.25
18:16
39.85
GKNe (32)
138.22
17.84
39.07
222.31
12.05
40.32
546.06
42.76
38.47
372.90
57.58
34.45
230:44
19.50
39.53
GiliNer 125]
143.6
18:35
38.76
225 66
12.70
39.91
565.47
38.28
37.51
360.05
54.33
34.44
225 30
19.46
38.90
LCS [45]
154.82
20.11
38.93
217.55
11.27
39.94
587.44
39.79
39.42
356
54.20
34.76
238.19
20.77
39.36
Ou AICT
133.85
16.50
40.55
206.34
10.74
40.68
594.92
41.27
37.93
328.45
42.58
35.35
204.67
16.53
39.99
In this section, we provide supplementary quantitative results to further demonstrate the advantages of our proposed method. First, we evaluate AICT against more state-of-the-ar methods using low- resolution test images from the iHarmony4 dataset [8]. Following previous work, we downsample these images to 256 x 256 pixels to obtain low-resolution versions. It is important to note that AICT employs parameters trained on high-resolution images from the iHarmony4 dataset [8] and uses low-resolution images as input to perform pixel-wise color transformation instead of downsampling high-resolution output images for comparison. As shown Table 7, AICT achieves superior performance across most metrics, which demonstrates the effectiveness of our method on low- resolution images. However, our method performs slightly worse on the HDay2night subset due to the limited number of training images in this subset.
We also compare AICT with state- of-the-art methods on the HAdobe5k subset at resolution of 1024 x 1024. As shown in Table 8, our method achieves the best performance in terms of MSE and PSNR, and per- forms slightly worse than CDTNet- 256 [7] in terms of fMSE and SSIM. is important to note that CDT- Net [7] requires retraining for im- ages of varying resolutions. In con- trast, AICT demonstrates flexibil- ity by using parameters trained on high-resolution images from the iHar- mony4 dataset [8] to effectively pro- cess images at various resolutions.
Table 8: Quantitative comparison the HAdobe5k subset at 1024 x 1024 resolution. The best results are marked in bold, and the second best are underlined.
Method
iMSEL
MSEJ
PSNR
SSIM
pix2pixHD [42]
332.43
63.45
31.64
0.9135
CRD [4]
259.28
90.11
29.77
0.8225
HiDT [1]
1501.93
265.32
29.95
0.9628
DoveNet [8]
312.88
51.00
34.81
0.9729
S²AM [9]
262.39
47.01
35.68
0.9784
IIH [15]
417.33
56.34
34.69
0.9471
RainNet [22]
305.17
42.56
36.61
0.9844
iS? AM [33]
168.85
25.03
38.29
0.9846
CDTNet-2266 [7]
152.13
21.24
38.77
0.9868
AdaInt [47]
235.86
31.75
37.81
0,9837
SepLUT [48]
224.99
27.41
37.75
0.9826
Harmonizer [20]
229.17
32.67
35.75
0,9081
In order to compare AICT with state- of-the-art methods ultra-high res- olutions, we collect two ultra high- resolution benchmarks (over4K and over5K) by selecting images with res- olutions exceeding 4K (4096 2160) and 5K (5120 2880) from the HAdobe5k subset. As shown in Table 9, our method achieves the best performance on the over4K benchmark in terms of IMSE, MSE, and SSIM, and the second-best performance in terms of SSIM. As shown in Table 10, on the over5K benchmark, our method achieves the best performance in terms of MSE and PSNR and the second- best performance in terms of fMSE and SSIM.
DCCF [46]
196.41
22.63
38.39
0.9846
PCT-Net [12]
171.04
22,57
39.39
0.9860
Our AICT
156.81
19.50
39.67
0.9864
14

---

We further compare the FLOPs, mem- ory cost, and inference time of our method with state-of-the-art methods on the HAdobe5k subset. These met- rics are crucial for evaluating the com- putational efficiency and scalability of the models. Our tests were conducted on computer equipped with 32GB of memory and GeForce GTX 1070 Ti GPU. Tables 11 and 12 show the comparison results at 1024 1024 and 2048 x 2048 resolutions, respectively. At both resolutions, Harmonizer [20] has significant advantage in terms of FLOPs, while DCCF [46] has the lowest memory cost. At resolution of 1024 x 1024, our method is comparable to PCT- Net [12] in FLOPs, memory cost, and inference time. However, method demonstrates lower memory cost and shorter inference time compared to PCT-Net. One important reason is that the parameter maps in our method have a fixed size, whereas PCT-Net needs to align the parameter maps with the composite images, resulting in higher memory consumption and longer inference time.
Table Quantitative comparison on the HAdobe5k subset
at resolutions exceeding 4K (4096 x 2160). The best results
are marked in bold, and the second best are underlined.
Method
IMSEL
MSEI
PSNRT
SSIM
AdaInt [47]
226.67
27.84
38.20
0.9882
SepLUT [48]
219.61
24.98
38.13
0.9866
Harmonizer [20]
201.44
23.06
37.89
0.9368
DCCF [46]
199.85
20.55
38.39
0.9885
PCT-Net [12]
165.31
19.36
39.83
0.9900
Our AICT
154.45
16.79
40.16
0.9898
resolution of 2048 x 2048, our
A.2 More Qualitative Results
We present additional qualitative com- parison results on the iHarrony4 dataset [8], shown in Figure 7.
Table 10: Quantitative comparison on the HAdobe5k subset at resolutions exceeding 5K (5120 x 2880). The best results
are marked in bold, and the second best are underlined.
Specifically,
compare posed method with Harmonizer [20],
pro-
Method
iMSE.
MSE.
PSNRT
SSIM
DCCF [46], and PCT-Net [12]. The qualitative results demonstrate that method achieves superior harmo- nization performance, generating im- ages that closely resemble the ground truth. Our method excels in adjusting the overall brightness and preserving local details in the foreground, leading to more realistic and visually pleasing composite images.
AdaInt [47]
330.25
29.26
36.35
0.9782
SepLUT [48]
395.42
22.06
36.30
0.9775
Harmonizer [20]
312.87
36.78
36.55
0.9437
DCCF [46]
320.03
28.44
35.73
0.9757
PCT-Net [12]
229.63
21.41
37.49
0.9790
Our AICT
293.09
15.49
37.64
0.9786
Table 11: Comparison of inference time, memory cost, and FLOPs 1024
1024 resolution.
Image Size
Method
FL.Pss] (G)
Memory,, (MB)
Inference Time. (ms)
Harmonizer [20]
0.036
1034
15.04
1024
DCCF [46]
12.68
889
37.66
PCT-Net [12]
13.04
1280
72.01
1024
Our AICT
14.35
1289
72.32
Table 12: Comparison of inference time, memory cost, and FLOPs 2048 2048 resolution.
Image Size
Method
FLOPs| (G)
Memoryl, (MB)
Inference Time. (ms)
Harmonizer [20]
0,036
1959
54.68
2048
DCCF [46]
12.68
957
36.94
PCT-Net [12]
13.04
2078
99.08
2048
Our AICT
14.35
1653
91.64
A.3
Evaluation on Real Composite Images
We provide qualitative results on real composite dataset [7], which presents significant challenge by combining foreground regions from different light fields with background regions. The resolution of this dataset ranges from 1024 x 1024 to 6016 x 4000 pixels. As shown in Figure 8, AICT outperforms other methods in producing visually superior results in real-world scenarios. The images generated by AICT show more harmonious appearance, with both overall brightness and local foreground
15

---

(x) Compusile image
(b) Harmonizer
(c) DCCF
(d) PcT-Net
[e) AICT
(0 Ground truth
Figure Qualitative comparison results and eтrоr maps. We visualize the error between the harmonized images and the ground truth images. The error maps normalized for display, and the foreground outlined in red.
Table 13: User study results. The best result is marked bold.
Method Harmonizer [20]
DCCF [46]
PCT-Net [12]
Our AICT
Scoret
21.60
28.00
32.40
37.80
details appearing more realistic. Following [39], we first randomly select 20 high-resolution real composite images from this dataset to conduct user study. Then, 20 volunteers independently rank
16

---

(a) Composite image
(b) Harmonizer
(c) DCCF
(d) PCT-Net
(e) AICT
Figure 8: Qualitative comparison results against state-of-the-art methods the real composite dataset. The foreground is outlined red.
the predictions from to 3 based on visual quality. Scores of 3, 2, and are assigned for ranks and 3, respectively. The mean scores for each method are presented in the table below. As shown in Table 13, our method achieves the highest score. The above experimental results demonstrate the superiority and generalizability of our method.
A.4 Broader Impacts
This paper proposes pixel-wise high-resolution image harmonization method aimed adjusting the color of the foreground to scamlessly integrate with the background, thereby enhancing the realism

---

of composite images and making significant contribution to the image composition community. Simultaneously, our research holds paramount importance in domains such as art, entertainment, and commerce. However, it is worth noting that image harmonization techniques could potentially be exploited to create deceptive or misleading visual content. To address this concern, there is pressing need for the development of image manipulation detection methods. Additionally, we emphasize the importance of raising awareness about the capabilities and limitations such methods to effectively mitigate their adverse impacts.
A.5 Limitations
primary limitation of our method lies in its incapability effectively address foreground elements like mirrors and glass. This is because the model cannot accurately simulate the interplay between light and various materials in such scenarios. Figure illustrates some failure cases where our method struggles to harmonize images containing such elements.
(8) Composite image
(b) AICT
(c) Ground truth
(a) Composite image
(b) AICT
(c) Ground bruth
Figure 9; Examples of failed harmonization the iHarmony4 dataset.
18

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The main claims made in the abstract and introduction accurately reflect the paper's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: The paper discusses the limitations of the work performed by the authors in Section A.5.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-specfiiatiinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
19

---

Answer:
[NA]
Justification:
The paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: The paper fully discloses all the information needed to reproduce the main experimental results of the paper in Section 3.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, descríbing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e-g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
5. Open access to data and code
20

---

The authors should provide scripts to reproduce all experimental results for the new reproducible, they Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: The data used are publicly available, and the code can be found at https: //g //githb... . com/ aipixel/AIQo /ICCT, along with sufficient instructions faithfully reproduce the main experimental results.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" an acceptable answer. Papers cannot be rejected simply for including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: / //ipsscc... wwtpccppcpgguupiieeecceeecooceee gwwd//////////gg Saoaaee///odSSmmaaacaaaa
onPolicy) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
proposed method and baselines. If only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permirted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The paper specifies all the training and test details in Section 4.1.2.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
The code can be found at httts:://ittiiiii com aapixelppleeeee/eee ppppxeee/AACT... /AICT. The results can be reproduced using the provided code and the implementation details described in the paper, so no error bars are reported in this paper.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
21

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: The paper provides sufficient information the computer resources in Sec- tion 4.1.2.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental rums as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurip:: http://uuuripscccpuuuu.pp cc/public//Bieccccccecceee Beteee/ttiihhheeeeeeeeee esGuidelines
Answer: [Yes]
Justification: The research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: The paper discusses both potential positive socictal impacts and negative societal impacts of the work performed in Section A.4.
Guidelines:
22

---

The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negatíve societal impacts include potential malicious or unintended uses (e.g.. disinformation, generating fake profiles, surveillance), fairness considerations (c.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of MI.).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All assets used in the paper, including code, data, and models, have been prop- erly credited to their original creators. The licenses are explicitly mentioned in Section 1.1.1 and 4.2.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
23

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperswi thcode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: The paper proposes a new model, and the documentation is provided in Section 3.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question:
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsoureing or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
24

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
25


Reviews:
[{"review_id": "tyP725Nbv1", "reviewer": "Reviewer_pwQP", "summary": "This paper proposes a new method called Adaptive-Interval Color Transformation (AICT) for high-resolution image harmonization. The key ideas are:\n\n1. Predicting pixel-wise color transformations using a parameter network that generates multiple 3D lookup tables (LUTs).\n2. Separating the color transform into cascaded sub-transformations using two 3D LUTs to adaptively adjust the sampling intervals and model local non-linearities.\n3. Using a global consistent weight learning module to predict image-level weights for each transformation to enhance overall harmony.\n\nExtensive experiments show that AICT achieves state-of-the-art performance on benchmark datasets while being computationally efficient.", "questions": "Please see weakness.", "limitations": "Please see weakness.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. **Novel approach to high-resolution image harmonization**: The paper proposes a new method, Adaptive-Interval Color Transformation (AICT), which addresses the challenging problem of harmonizing high-resolution composite images. The key idea of using adaptive sampling intervals in the color transformation is a novel contribution that sets AICT apart from prior works.\n\n2. **Improved quantitative performance on benchmark datasets**: AICT achieves state-of-the-art results on the widely-used iHarmony4 and HCOCO datasets, outperforming previous methods in terms of fMSE, MSE, and PSNR metrics. These quantitative improvements, although relatively small in some cases, demonstrate the effectiveness of the proposed technical innovations.\n\n3. **Computational efficiency and scalability**: The paper presents a detailed analysis of the computational complexity and memory usage of AICT, showing that it achieves a favorable trade-off between performance and efficiency compared to existing high-resolution harmonization methods. The scalability of AICT to higher resolutions (e.g., 2048x2048) is also demonstrated, which is important for practical applications.\n\n4. **Insights into the role of adaptive sampling intervals**: Through ablation studies and visualizations, the paper provides some insights into the importance of adaptive sampling intervals in the color transformation process. The comparison between fixed and adaptive intervals highlights the benefits of the proposed approach in terms of handling local color variations and edge artifacts.", "weaknesses": "## Unrealistic Settings\nThe input resolutions in the main experiments (1024x1024 and 2048x2048) are still relatively low compared to professional compositing workflows, which often deal with 4K or higher resolutions. The effectiveness of AICT for ultra high-resolution images is unclear and should be validated with appropriate benchmarks.\n\n## Lack of Comparison to Extensively Studied Prior Art: Proposed Approach Appears to be a Trivial Amalgamation of Established Techniques\n- The use of 3D LUTs for color transformations is not a fundamentally new idea in image processing. Many previous works, especially in the photo enhancement domain, have used similar techniques. The paper does not sufficiently explain how AICT's specific LUT-based approach is distinct from prior methods.\n- Adaptively adjusting the sampling intervals of color transformations has also been explored in works like AdaInt. While AICT's dual cascaded LUTs are a nice extension, this still seems like an incremental contribution building on existing ideas.\n- The global weight learning module likewise appears very similar to the global adjustment parameters used in existing works. More discussion is needed on how AICT's weights differ and what additional capabilities they enable.\n\n## Missing motivations for method modules\n\n- The global weight learning module seems disconnected from the rest of the method. It's unclear why weighting the predicted LUTs based on global image features would improve harmonization quality. The authors should explain the reasoning behind this design choice and how it relates to the adaptive interval learning.\n- The global weight learning module predicts a single scalar weight for each LUT, which is then uniformly applied to all pixels. This global approach contradicts the motivation of AICT to enable local color adjustments. A more spatially-varying weighting scheme would be more consistent with the overall goal of the method.\n\n\n## Evaluation\n\n- The use of cascaded dual 3D LUTs is a key aspect of AICT, but is not ablated. Testing a single LUT baseline, more numbers of LUTs or alternative compositions of LUTs would help validate the necessity of the proposed decomposition. \n- The user study results are not reported, making it difficult to assess AICT's perceptual quality compared to other methods. Quantitative metrics like fMSE and PSNR do not always correlate well with human judgments, so user ratings would provide valuable additional evidence for AICT's benefits.\n\n## Performance\nWhile the method outperforms previous approaches, the gains on some metrics (e.g. PSNR) are relatively modest (<1 dB). It would be good to see more analysis on what types of images/scenarios benefit most from AICT.\n\n\nOverall, the reviewer acknowledges that this paper addresses an important topic using generally reasonable methods. The initial score will be increased if the rebuttal effectively addresses the concerns mentioned above.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Thanks for your positive evaluation and valuable suggestions.**\n\n***Q1: The effectiveness of AICT for ultra high-resolution images is unclear and should be validated with appropriate benchmarks.***  \n***A1:*** Thanks for this suggestion. In Table 1 of the paper, we have reported the results  on the HAdobe5k  dataset with resolutions ranging from 312 × 230 to 6048 × 4032. Following the reviewer's suggestion, we collect two ultra high-resolution benchmarks (over4K and over5K) by selecting images with resolutions exceeding 4K (4096 × 2160) and 5K (5120 × 2880) from the HAdobe5k dataset. The compared results on the table below show that our method achieve the best performance on the over4K benchmark in terms of all metrics.  On the over5K benchmark, our method achieves the best performance in terms of MSE and PSNR and the second-best performance in terms of fMSE and SSIM.\n\n|Benchmark|Method|fMSE↓|MSE↓|PSNR↑|SSIM↑|\n|----|:----:|:----:|:----:|:----:|:----:|\n|  over4K  |Harmonizer|199.68|22.87|37.91|0.9374|\n||DCCF|199.85|20.54|38.38|0.9884|\n||PCT-Net|165.31|19.36|39.82|**0.9900**|\n||AdaInt|226.66|27.83|38.20|0.9882|\n||SepLUT|219.60|24.98|38.13|0.9866|\n||Our AICT|**154.44**|**16.79**|**40.16**|**0.9900**|\n\n| benchmark|Method|fMSE↓|MSE↓|PSNR↑|SSIM↑|\n|----|:----:|:----:|:----:|:----:|:----:|\n|over5K|Harmonizer|302.43|35.09|36.65|0.9441|\n||DCCF|320.02|28.44|35.73|0.9757|\n||PCT-Net|**292.62**|*21.40*|*37.48*|**0.9790**|\n||AdaInt|330.25|29.26|36.35|0.9781|\n||SepLUT|395.42|22.06|36.30|0.9774|\n||Our AICT|*293.08*|**15.48**|**37.64**|*0.9786*|\n\n***Q2 & Q3: Explain how AICT's specific LUT-based approach is distinct from prior methods.***  \n***A2 & A3:*** Compared with prior methods, our method has three distinct differences. **Firstly**, unlike previous methods that use fixed weight parameters to fuse pre-trained LUTs, our method dynamically predicts entire LUTs based on the input. **Secondly**, Previous methods use LUTs for global RGB-to-RGB transformations, which often overlook local context. In contrast, our method uses each pixel’s color and position as inputs for LUTs, enabling pixel-wise color transformations. **Thirdly**, previous methods apply adaptive interval learning and separable lookup tables to each lattice dimension of the 3D LUTs for global sampling interval adjustment, which limits the expressiveness of the 3D LUTs. In contrast, our method achieves pixel-wise sampling interval adjustment to model local non-linearities of the color transformation.\n\n***Q4 & Q5 & Q6: More discussion is needed on how AICT's weights differ and what additional capabilities they enable.***  \n***A4 & A5 & A6:*** Thanks for this suggestion. When a foreground is inserted into a new background, its average brightness is influenced by the background lighting. Predicting spatially consistent weights for global color adjustment can enhance adaptive interval learning. To further discuss AICT's weights, we conduct ablation studies by sequentially removing the parameters: first for the R channel ('w/o R'), then for both R and G channels ('w/o RG'), and finally all parameters ('w/o weight'). We also explore pixel-level weights ('spatial'). As shown in the first table below, increasing the number of removed parameters decreases performance, highlighting the role of weight learning module. The results of \"spatial\" show that learning spatially-varying weights does not enhance performance, as LUTs handle pixel-wise color transformations. To illustrate the role of the weight learning module in global color adjustment, we calculate the error between the average color values of harmonized and ground truth images. As shown in the second table below, our method performs better.\n\n|Method|fMSE↓|MSE↓|PSNR↑|SSIM↑|\n|----|:----:|:----:|:----:|:----:|\n|w/o R|230.83|18.25|39.39|**0.9910**|\n|w/o GB|232.20|19.10|39.18|0.9909|\n|w/o weight|247.48|19.40|39.16|0.9908|\n|spatial|230.24|18.03|39.23|0.9909|\n|our AICT|**228.43**|**17.76**|**39.40**|**0.9910**|\n\n|Method|w/o weight|our AICT|\n|----|:----:|:----:|\n|error↓|6.94|**6.57**|\n\n***Q7: Cascade LUTs need ablation studies***  \n***A7:*** Following this suggestion, we report more results in the table below. For each color channel, we use only 1 LUT for color transformation ('Single') and cascade 2 and 3 LUTs for adaptive interval learning ('Int x 2', 'Int x 3') and color transformation ('Tra x 2', 'Tra x 3'). We also cascade 4 and 6 LUTs for alternating adaptive interval learning and color transformation ('Alt x 2', 'Alt x 3'). Performance improves slightly with more LUTs but plateaus at three, due to increased training difficulty.\n\n|Method|fMSE↓|MSE↓|PSNR↑|SSIM↑|\n|----|:----:|:----:|:----:|:----:|\n|Single |248.51|19.71|39.15|0.9908|\n|Int x 2|227.95|**17.40**|**39.50**|**0.9910**|\n|Int x 3|229.50|18.55|38.95|0.9909|\n|Tra x 2|**228.40**|*17.50*|*39.41*|0.9909|\n|Tra x 3|230.28|19.01|37.95|0.9908|\n|Alt x 2|230.58|20.25|38.02|0.9908|\n|Alt x 3|232.25|19.89|37.80|0.9907|\n|our AICT|*228.43*|17.76|39.40|**0.9910**|\n\n***Q8: The user study results are not reported.***  \n***A8:*** Thanks for this suggestion. We randomly select 20 high-resolution real composite images from the GiftNet dataset. In the user study, 20 volunteers independently rank the predictions from 1 to 3 based on visual quality. Scores of 3, 2, and 1 are assigned for ranks 1, 2, and 3, respectively. The mean scores for each method are presented in the table below. As we can see our method achieves the highest score.\n\n|Method|Harmonizer|DCCF|PCT-Net|our AICT|\n|----|:----:|:----:|:----:|:----:|\n|Score↑|21.6|28|32.4|**37.8**|\n\n***Q9: More analysis on what types of images/scenarios benefit most from AICT.***  \n***A9:*** We present more results in Figure 2 in the PDF attached in the author rebuttal. As we can see that our AICT method particularly benefits images with rich textures, such as fur and hair. This is because AICT achieves non-uniform sampling intervals, allowing for better modeling of local non-linearities in color transformations.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "srfP66xsQy", "reviewer": "Reviewer_dAiu", "summary": "This paper presents a new method for harmonizing the color of foreground objects added to scenes with the colors of the background original image. The authors focus on a model able to present good results on high-resolution images. The idea is to learn two different sets of Look-Up-Tables. The first set aims at modifying the input values to re-scale them in a way that the second set can perform pixel-wise editing. The authors show the ability of their method in three different datasets.", "questions": "First, please address the weaknesses mentioned before.\n\nAlso, when reading this paper, I got the feeling that: \"This idea is mostly AdaInt + Mask\". I may have lost some important component here, so please guide me through why my feeling may not be correct.", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- The general idea of using two LUT-like transformations is interesting.\n- Results outperform the state of the art.", "weaknesses": "- The paper is difficult to follow at the beginning, as the authors abuse notation regarding 3DLUTs in color image processing. In color image processing 3DLUTs are a function that goes from R^3 to R^3, i.e. given an input (R,G,B) values they output a (R',G',B') value ---see for example [1]---. Therefore, in color image processing 3DLUTs cannot be approximated by curves (contrary to what is said by the authors in lines 138-139). What the authors are doing is learning spatial-aware 1D LUTs, i.e. a function from R^3 to R that given an input (x,y,R) outputs a value R'. It is true that some papers (citation 24 on the paper) already abuse notation when speaking of 4DLUTs. However, in their case, they are not redefining an already existing term, as it is 3DLUTs. Therefore, the authors of this manuscript need to rewrite different parts of the paper refraining from using the term 3DLUT to define their approach. As it currently stands, it cannot be accepted due to this conceptual error.\n\n- Metrics are insufficient. MSE and PSNR are strongly correlated metrics. In the end, PSNR=10log_{10}(MAX^2/MSE), where MAX is the maximum possible value. Therefore, the authors need to add further metrics such as LPIPS, DeltaE as perceptually-based full-referenced metrics, and NIQE as a non-reference one.\n\n- The paper will be of much larger interest if the authors present the results of a user study, for example as it was done in PCT-Net (reference 12 in this paper) or in [2]. \n\n- Figures 1,3, and 4 are versions of the same general idea of Figure. It will be beneficial for the paper if the authors can present just 1 Figure in which all the small differences are included.\n\n[1] Conde, M. V., et al. Nilut: Conditional neural implicit 3d lookup tables for image enhancement, AAAI 2024\n[2] Valanarasu, J. M. J., et al. (2022). Interactive portrait harmonization, ICLR 2023", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Thanks for your positive evaluation and valuable suggestions.**\n\n***Q1:The authors abuse notation regarding 3DLUTs in color image processing.***  \n***A1:*** We thank the reviewer very much for pointing out the inaccurate usage of the term 3DLUTs. Following the reviewer's suggestion,  we will rename the term of 3DLUTs  to define our approach and rewrite the corresponding parts of the paper.\n\n***Q2: Metrics are insufficient. The authors need to add further metrics such as LPIPS, and DeltaE as perceptually-based full-referenced metrics, and NIQE as a non-reference one.***  \n***A2:*** Thanks for this suggestion. Following this suggestion, we use LPIPS, DeltaE, and NIQE metrics to evaluate the  performance of the compared methods on the  iHarmony4 dataset. The table below shows the quantitative results.   As we can see that our method achieves the best results in terms of LPIPS and DeltaE and the third best results in terms of NIQE. It should be noted that NIQE is a no-reference metric that does not assess color or semantic context continuity. Therefore, it is not suitable for evaluating the performance of image harmonization.\n\n|Method|LPIPS↓|DeltaE↓|NIQE↓|\n|----|:----:|:----:|:----:|\n|Harmonizer|0.016|0.88|4.48|\n|DCCF|0.017|0.80|**4.46**|\n|PCT-Net|**0.013**|*0.67*|*4.47*|\n|our AICT|**0.013**|**0.66**|4.48|\n\n***Q3: The paper will be of much larger interest if the authors present the results of a user study, for example as it was done in PCT-Net (reference 12 in this paper) or in [2].***  \n***A3:*** Thanks for this suggestion. We randomly select 20 high-resolution real composite images from the GiftNet dataset. In the study, 20 volunteers independently rank the predictions from 1 to 3 based on visual quality, considering color and luminance consistency. Scores of 3, 2, and 1 are assigned for ranks 1, 2, and 3, respectively. The mean scores for each method are presented in the table below. As we can see our method achieves the highest score.\n\n|Method|Harmonizer|DCCF|PCT-Net|our AICT|\n|----|:----:|:----:|:----:|:----:|\n|Score↑|21.6|28|32.4|**37.8**|\n\n***Q4: Figures 1,3, and 4 are versions of the same general idea of Figure. It will be beneficial for the paper if the authors can present just 1 Figure in which all the small differences are included.***  \n***A4:*** Thanks for this suggestion. We will combine Figures 1, 3, and 4 into a single comprehensive figure that highlights all the small differences in the camera-ready version. \n\n***Q5: Also, when reading this paper, I got the feeling that: \"This idea is mostly AdaInt + Mask\". I may have lost some important component here, so please guide me through why my feeling may not be correct.***  \n***A5:*** We are sorry that our presentation let the reviewer misunderstand that our idea is mostly AdaInt + Mask. In fact,  our method is significantly distinct from AdaInt in three aspects. **Firstly**, AdaInt predicts only a few weight parameters to fuse several fixed LUTs obtained through training, which limits its expressive capacity. In contrast, our method dynamically predicts entire LUTs based on the input, resulting in significantly greater expressive power. **Secondly**, AdaInt uses LUTs to perform global RGB-to-RGB transformations, which overlook diverse local contexts. In contrast, our method uses the color and position of each pixel as inputs for the LUTs to perform pixel-wise color transformations. **Thirdly**, AdaInt applies adaptive interval learning to each lattice dimension of the 3D LUTs for global sampling interval adjustment, which limits the expressiveness of the 3D LUTs. In contrast, our method achieves pixel-wise sampling interval adjustment to model local non-linearities of the color transformation.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "lM8stkjQio", "reviewer": "Reviewer_4Tt1", "summary": "This paper proposes an AdaptiveInterval Color Transformation method (AICT) for high-resolution image harmonization, which predicts pixel-wise color transformation and adaptively adjusts the sampling interval to model local non-linearities of the color transformation at high resolution.", "questions": "Please address the questions in \"Weakness\".", "limitations": "See the \"Weakness\".", "rating": 7, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper is overall well-written and easy to understand.\n2. The idea is neat and clean.\n3. The experimental results look good.", "weaknesses": "1. The authors should conduct experiments using the training/test set of HAdobe5k which focuses on high-resolution, and report the results under different resolutions (1024, 2048, etc). More baselines should be compared in this setting.\n2. The authors should test on high-resolution real composite images and conduct user study.\n3. The authors are suggested to evaluate the method on ccHarmony dataset, which can reflect the illumination variation more faithfully.\n4. The authors should discuss the limitation of the proposed method and show some failure cases.\n5. More efficiency analyses (FLOPs, memory, time) should be provided.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Thanks for your positive evaluation and valuable suggestions.**\n\n***Q1: The authors should conduct experiments using the training/test set of HAdobe5k which focuses on high-resolution, and report the results under different resolutions (1024, 2048, etc). More baselines should be compared in this setting.***  \n***A1:*** Thanks for this suggestion. We have reported the results under the  resolutions 1024x1024 and 2048x2048 using the training/test set of HAdobe5k in Tables 2 and 7 of the submitted manuscript. Following this suggestion, we present comparison with two additional baselines:  AdaInt [1] and SepLUT [2]. As shown in the tables below, our method outperforms AdaInt and SepLUT in terms of all metrics. \n\n|Image Size|Method|fMSE↓|MSE↓|PSNR↑|SSIM↑|\n|----|:----:|:----:|:----:|:----:|:----:|\n|1024 x 1024|AdaInt|235.86|31.75|37.80|0.9837|\n||SepLUT|224.98|27.41|37.75|0.9826|\n||Our AICT|**156.81**|**19.50**|**39.67**|**0.9863**|\n\n|Image Size|Method|fMSE↓|MSE↓|PSNR↑|SSIM↑|\n|----|:----:|:----:|:----:|:----:|:----:|\n|2048 x 2048|AdaInt|221.90|31.09|38.10|0.9846|\n||SepLUT|216.14|27.02|37.97|0.9834|\n||Our AICT|**147.99**|**17.92**|**40.07**|**0.9871**|\n\n[1] Yang, Canqian, et al. AdaInt: Learning adaptive intervals for 3D lookup tables on real-time image enhancement. CVPR 2022.\n\n[2] Yang, Canqian, et al. Seplut: Separable image-adaptive lookup tables for real-time image enhancement. ECCV 2022.\n\n***Q2: The authors should test on high-resolution real composite images and conduct user study.***  \n***A2:*** Thanks for this suggestion. We test on a high-resolution real composite image dataset [3] with resolutions ranging from 1024 × 1024 to 6016 × 4000. We randomly select 20 images for the user study. In the study, 20 volunteers independently rank the predictions from 1 to 3 based on visual quality, considering color and luminance consistency. Scores of 3, 2, and 1 are assigned for ranks 1, 2, and 3, respectively. The mean scores for each method are presented in the table below. As we can see that our method achieves the highest score.\n\n|Method|Harmonizer|DCCF|PCT-Net|our AICT|\n|----|:----:|:----:|:----:|:----:|\n|Score↑|21.6|28|32.4|**37.8**|\n\n[3] Li Niu, et al. Deep image harmonization with globally guided feature transformation and relation distillation. ICCV 2023.\n\n***Q3: The authors are suggested to evaluate the method on ccHarmony dataset, which can reflect the illumination variation more faithfully.***  \n***A3:*** Thanks for this suggestion. Actually, we have evaluated the method on the ccHarmony dataset and presented the results in Table 3 of the submitted manuscript. Our method achieves the best results in terms of MSE, PSNR, and SSIM metrics, which demonstrates the effectiveness of the proposed method when handling illumination variation.\n\n***Q4: The authors should discuss the limitation of the proposed method and show some failure cases.***  \n***A4:*** Thanks for this suggestion. We have discussed the limitations of the proposed method and shown some failure cases in Section A.4 of the supplementary materials of the submitted paper.\n\n***Q5: More efficiency analyses (FLOPs, memory, time) should be provided.***  \n***A5:*** Thanks for this suggestion. In the submitted manuscript, we have provided the FLOPs, memory cost, and inference time on the HAdobe5k dataset for different resolutions (1024 x 1024 and 2048 x 2048). Please refer to Tables 8 and 9 of the submitted manuscript.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FgXxQa8MBP", "reviewer": "Reviewer_M5vB", "summary": "This work proposes the Adaptive-Interval Color Transformation (AICT) method for harmonizing high-resolution composite images. The proposed AICT first uses a parameter network to predict multiple curves (as 3D LUTs) to perform pixel-wise color transformation at low-resolution. The AICT then adjusts the sampling intervals of the color transformation by splitting each predicted 3D LUT into two 3D LUTs. Last, AICT uses a global consistent weight learning method to predict an image-level weight for each color transform. Experiments on the iHarmony4 [8] and ccHarmony [26] datasets show promising quantitative results of the proposed AICT.", "questions": "Q1: What is the novelty of the proposed 3D LUTs-based adaptive-interval color transformation, compared to the non-uniform sampling interval learning in Yang et al. [47] and the decomposition of a single color transform into two sub-transformations in Yang et al. [48]. It is suggested to explain the challenges of directly combining [47] and [48], and how the proposed AICT addresses such challenges. Comparison results between AICT and a simple combination of [47] and [48] are suggested to provide.\nQ2: Can authors report and explain the results of incorporating more 3D LUTs?\nQ3: According to Tables 4 and 5, the foreground-normalized MSE loss [34] is more important to the final results compared to the ``w/o Weight’’ and ``w/o Color’’. Can authors explain this and/or provide more results for clarification?\nQ4: Any more ablations on the proposed ``global weight’’?\nQ5: Can authors report non-reference image quality metrics (e.g., NIQE and LPIPS) for the harmonized real composite images in Figure 9?\nQ6: Can authors provide intermediate visual results of the ablated models?\nQ7: Can authors explain why using the channel-crossing strategy does not improve the performance?\nQ8: Any reason not to use SSIM as the evaluation metric?", "limitations": "The paper has discussed the limitations and potential negative societal impact. I cannot think of any further critical points.", "rating": 5, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Originality: \n(+) The strength is that the difference to existing HR image harmonization methods is clear.\n\nQuality: \n(+) The proposed method is complete and technically sound. (+) The experiment settings (evaluation datasets, metrics, and competing methods) are convincing.\n\nClarity: \n(+) The paper presents sufficient implementation details for reproducing the proposed method.\n\nSignificance: \n(+) Quantitative results in Tables 1-3, and 6-7 show superior or comparable performance of AICT in harmonizing HR and LR composite images.", "weaknesses": "Originality: \nThis work proposes the AICT method to address an existing task of harmonizing high-resolution (HR) composite images. While previous HR image harmonization methods (e.g., [12,46]) predict and then upsample low-resolution (LR) parameter maps for harmonization, the proposed AICT also predicts LR parameter maps but applies two 3D LUTs to harmonize the colors in a coarse-to-fine manner. (-) One weakness is the limited/vague novelty of using two 3D LUTs for color transformation, which is a combination of ideas from existing LUT-based image enhancement methods [47, 48] but lacks necessary discussions and experimental verifications.\n\nQuality: \n(-) The paper mentions the ``non-linearities of the color transform at high resolution’’ many times as the key problem suffered by existing methods and addressed by the proposed AICT. However, the paper does not explain what the ``non-linearities’’ are, nor do the visual results in Figures 7, 8, and 9 illustrate such advantages.\n(-) The paper claims AICT being lightweight and computationally efficient but Figure 2 only demonstrates that AICT has a small model size. According to Tables 8 and 9, AICT runs much slower than [21] and [46] and is comparable to [12]. Hence, such a claim needs to be tuned down.\n(-) The paper claims the ``global consistent weight method’’ to be novel but the paper does not explain its novelty.\n\nClarity: \nThere are a few weak points regarding the paper writing. \n(-) The paper mentions the ``adaptive interval learning method’’ and the ``global consistent weight method’’ as the second main contribution (line 86-87, page 2). However, the Method section discusses the proposed ``global consistent weight method’’ very briefly as a part of subsection 3.2 (line 207-214, page 5). The authors are suggested to either explain the ``global consistent weight method’’ in detail regarding its novelty or remove the corresponding claim in the second contribution.\n(-) According to Figure 3 and Section 3 (line 134-144, page 4) the LR branch of the proposed AICT outputs two parameter maps (C and F). However, Eq. (3) indicates that there is a low-resolution harmonized image predicted by the LR branch.\n(-) Figure 1 is not self-contained. The S2CRNet [22] is missing in Figure 2.\n(-) The sub-title of Section 2.1 is better: ``High-resolution Image Harmonization.\n(-) There are no discussions on the relations between the proposed AICT and existing methods in Section 2.1 and 2.2.\n(-) There are many repeated sentences in, for example,(Line 5-6, Line 60-63, Line 374-378), and (Line 7-13, Line 67-78).\n(-) The references contain redundant items, such as [13,14].\n(-) ``M=6’’ and ``M=10’’ should be ``K=6’’ and ``K=10’’.\n(-) It is better to move the Tables 6 and 7 to the main paper.\n\nSignificance: \n(-) The visual results in Figure 7-9 do not show significant improvements.\n(-) The main quantitative performance gain seems to come from the foreground-normalized MSE loss [34], according to Tables 4 and 5.\nThe paper presents a harmonization method that can produce better quantitative results, but the visual advantages illustrated in this paper are not significant", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Thanks for your positive evaluation and valuable suggestions.** \n\n***W1 & Q1: Limited novelty of using two 3D LUTs for color transformation, which is a combination of ideas from existing LUT-based image enhancement methods [47, 48] but lacks necessary discussions and experimental verifications..***  \n***R1 & A1:*** Regarding the novelty of using two 3D LUTs for color transformation, we request the reviewer to refer to our response to Q2 & Q3 of Reviewer pwQP due to the space limitation.\n\nOur method is not a combination of  Adaint [47] and Seplut [48]. In fact, simply combining them faces the challenges of limited color transformation capability and overlooking diverse local contexts. Our AICT addresses these challenges by dynamically predicting entire LUTs for pixel-wise sampling interval adjustment and color transformations.\nWe also conduct experimental comparison between our method and Adaint and Seplut as well as their combination. As shown in the table below, our AICT achieves the best performance in terms of all metrics.\n\n|Method|fMSE↓|MSE↓|PSNR↑|SSIM↑|\n|----|:----:|:----:|:----:|:----:|\n|Adaint|358.28|31.96|37.41|0.9886|\n|Seplut|358.51|31.35|37.33|0.9876|\n|Adaint+Seplut|252.32|30.55|37.66|0.9889|\n|our AICT|**228.43**|**17.76**|**39.40**|**0.9910**|\n\n***W2: Explain “the non-linearities”  and show the visual results.***  \n***R2:*** Due to resolution differences,  predicted 3D LUTs cannot align with the original input image. Consequently, linear interpolation based on a few sampling points is performed within the 3D LUT to obtain output color values, leading to \"non-linear\" transformations. To illustrate this advantage, we present the color distribution in local areas of high-resolution harmonized and ground truth images in Figure 3 of the PDF attached in the author rebuttal, which shows that the pixel values predicted by our method are closer to the real color distribution.\n\n***W3: “lightweight and computationally efficient” needs to be tuned down.***  \n***R3:*** Thanks for this suggestion. We will tune down our claim in the revised manuscript.\n\n***W4 & Q4: Explain novelty of the 'global consistent weight method'.***  \n***R4 & A4:*** We request the reviewer to refer to our response to Q4 & Q5 & Q6 of the Reviewer pwQP due to space limitation.\n\n***W5: There are a few weak points regarding the paper writing.***  \n***R5:*** Thanks for this suggestion. We will improve the paper writing in the revised manuscript. \n\n***W6: The visual results do not show significant improvements.***  \n***R6:*** To better show significant improvements, we quantify and visualize the error between the harmonized and ground truth images as shown in Figure 1 of the PDF attached in the author rebuttal. We also present additional visual results in Figure 2 of the PDF.\n\n***Q2: Report and explain the results of incorporating more 3D LUTs.***  \n***A2:*** We report more results in the table below. For each color channel, we use only 1 LUT for color transformation ('Single') and cascade 2 and 3 LUTs for adaptive interval learning ('Int x 2', 'Int x 3') and color transformation ('Tra x 2', 'Tra x 3'). We also cascade 4 and 6 LUTs for alternating adaptive interval learning and color transformation ('Alt x 2', 'Alt x 3'). Performance improves slightly with more LUTs but plateaus at three, due to increased training difficulty.\n\n|Method|fMSE↓|MSE↓|PSNR↑|SSIM↑|\n|----|:----:|:----:|:----:|:----:|\n|Single |248.51|19.71|39.15|0.9908|\n|Int x 2|227.95|**17.40**|**39.50**|**0.9910**|\n|Int x 3|229.50|18.55|38.95|0.9909|\n|Tra x 2|**228.40**|*17.50*|*39.41*|0.9909|\n|Tra x 3|230.28|19.01|37.95|0.9908|\n|Alt x 2|230.58|20.25|38.02|0.9908|\n|Alt x 3|232.25|19.89|37.80|0.9907|\n|our AICT|*228.43*|17.76|39.40|**0.9910**|\n\n***Q3: Explain why foreground-normalized MSE loss [34] is more important***  \n***A3:*** The foreground-normalized MSE loss, widely used in methods like PCT-Net and DCCF, prevents instability in images with very small objects. For ablation studies, we set Amin to 100, 5000, and 10000, denoted as 'Amin=100', 'Amin=5000', and 'Amin=10000'. In our AICT, Amin is set to 1000. As shown in the table, when Amin is 100, performance doesn't improve due to the scarcity of smaller targets. Increasing Amin from 1000 to 10000 causes instability in training on small targets, resulting in decreased performance.\n\n|Method|fMSE↓|MSE↓|PSNR↑|SSIM↑|\n|----|:----:|:----:|:----:|:----:|\n|MSE|287.04|23.17|38.41|0.9908|\n|Amin=100|229.87|17.96|39.23|0.9909|\n|Amin=5000|234.22|20.33|38.69|0.9852|\n|Amin=10000|250.89|22.56|37.56|0.9864|\n|our AICT|**228.43**|**17.76**|**39.40**|**0.9910**|\n\n***Q5: Report non-reference image quality metrics (e.g., NIQE and LPIPS) for Figure 9.***  \n***A5:*** Since LPIPS is a full-reference metric, we only report NIQE in the table below. Our method obtains the best results in the first two examples and the second-best result in the last example. \n\n|Method|Composite image|Harmonizer|DCCF|PCT-Net|our AICT|\n|----|:----:|:----:|:----:|:----:|:----:|\n|NIQE↓|10.51|9.18|*9.09*|9.35|**9.06**|\n|NIQE↓|2.85|*2.82*|2.82|2.82|**2.82**|\n|NIQE↓|3.21|3.15|3.16|**3.15**|*3.15*|\n\n***Q6: Provide intermediate visual results of the ablated models***  \n***A6:*** The intermediate visual results are shown in Figure 4 of the PDF attached in the author rebuttal.\n\n***Q7: Explain why the channel-crossing strategy fails to improve performance***  \n***A7:*** Our parameter model uses an RGB image to predict parameter maps, which inherently include information from all color channels. The channel-crossing strategy introduces redundancy without improving performance.\n\n***Q8: Use SSIM as the evaluation metric***  \n***A8:*** SSIM is actually not a commonly used metric for image harmonization tasks. Following this suggestion, we have used SSIM as an evaluation metric. As shown in the table, our AICT achieves a competitive SSIM score of 0.9910.\n\n|Method|Harmonizer|DCCF|PCT-Net|our AICT|\n|----|:----:|:----:|:----:|:----:|\n|SSIM↑|0.9685|0.9896|**0.9911**|0.9910|", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
Dear Reviewers,

We would like to express our sincere gratitude to all the reviewers for their constructive feedback and for recognizing the performance and efficiency of our proposed method. We appreciate your valuable suggestions and have carefully addressed each point in our responses.

1.**Reviewer pwQP (Score: 5 - Borderline Accept)** recognizes the novel aspects of our AICT method and its computational efficiency. We have addressed your concerns by validating our approach on ultra high-resolution images and clarifying its distinctiveness from prior methods. Additionally, we have provided detailed explanations for the global weight learning module and included extensive ablation studies. 

2.**Reviewer dAiu (Score: 4 - Borderline Reject)** acknowledges the innovative use of LUT transformations and the performance of our method compared to state-of-the-art techniques.  Following the reviewer's suggestion, we will rename the term of 3DLUTs to define our approach and rewrite the corresponding parts of the paper. We have added LPIPS, DeltaE, and NIQE metrics to our evaluation, demonstrating the superiority of our method. A user study has been conducted to further validate our results. Additionally, we have highlighted the key differences between our method and AdaInt to address your concerns.

3.**Reviewer 4Tt1 (Score: 7 - Accept)** acknowledges the clarity and the effectiveness of our method. In response to your suggestions, we have conducted additional experiments using the HAdobe5k dataset at various resolutions (1024x1024 and 2048x2048), compared our method with more baselines, and reported the results, demonstrating superior performance in terms of MSE and PSNR. We have also evaluated our method on high-resolution real composite images and the ccHarmony dataset, and discussed method limitations and failure cases in detail. Furthermore, we expand our efficiency analysis to include FLOPs, memory usage, and inference time for different resolutions.

4.**Reviewer M5vB (Score: 5 - Borderline Accept)** praises the neat and clean idea as well as the impressive experimental results. We have clarified the novelty of the 3D LUTs-based approach and demonstrated the advantages of AICT through comparative experiments. We address the issues related to non-linearities and efficiency, and adjust our claims accordingly. Additionally, we expand the discussion on the global consistent weight method, conduct further ablation studies, and correct errors and redundancies in the text.

Our responses to individual comments  of each reviewer are posted in the rebuttal under each reviewer's report.  All the required experimental results are presented in the PDF attached in  this rebuttal. 

Specifically:

- **Figure 1**  visualizes the error between the harmonized and ground truth images.

- **Figure 2**  presents qualitative results against existing methods.

- **Figure 3**  displays the color distribution in local areas of high-resolution harmonized and ground truth images, showing that the pixel values predicted by our method are closer to the real color distribution.

- **Figure 4**  presents the visual results of  ablated models.

For convenience, we highlight the figure relevant to each reviewer's comments as follows:

- **Reviewer pwQP**: Figure 2.
﻿
- **Reviewer M5vB**: Figure 1, Figure 2, Figure 3, and Figure 4

We hope that our responses and the revisions made to the manuscript will alleviate any concerns and enhance the overall quality of our submission. Once again, we thank all the reviewers for their thoughtful reviews and valuable suggestions.

Meta-review: This paper proposes a method for high-resolution image harmonization. It predicts pixel-wise color transformations using a parameter network that generates multiple 3D lookup tables (LUTs). It separates the color transform into cascaded sub-transformations using the LUTs. And a global consistency is proposed for the optimized performance. After the rebuttal, all the reviewers find their concerns have been well addressed and give positive recommendation for this submission.
Decision: Accept (poster)