Title: High-Resolution Image Harmonization with Adaptive-Interval Color Transformation
Venue: NeurIPS 2024
Accepted: yes
Keywords: Image Composition

Image Harmonization

Lookup Table
Abstract: Existing high-resolution image harmonization methods typically rely on global color adjustments or the upsampling of parameter maps. However, these methods ignore local variations, leading to inharmonious appearances. To address this problem, we propose an Adaptive-Interval Color Transformation method (AICT), which predicts pixel-wise color transformations and adaptively adjusts the sampling interval to model local non-linearities of the color transformation at high resolution. Specifically, a parameter network is first designed to generate multiple position-dependent 3-dimensional lookup tables (3D LUTs), which use the color and position of each pixel to perform pixel-wise color transformations. Then, to enhance local variations adaptively, we separate a color transform into a cascade of sub-transformations using two 3D LUTs to achieve the non-uniform sampling intervals of the color transform. Finally, a global consistent weight learning method is proposed to predict an image-level weight for each color transform, utilizing global information to enhance the overall harmony. Extensive experiments demonstrate that our AICT achieves state-of-the-art performance with a lightweight architecture. The code is available at https://github.com/aipixel/AICT.

High-Resolution Image Harmonization with Adaptive-Interval Color Transformation

Abstract

Existing high-resolution image harmonization methods typically rely on global color adjustments or the upsampling of parameter maps. However, these methods ignore local variations, leading to inharmonious appearances. To address this problem, we propose an Adaptive-Interval Color Transformation method (AICT), which predicts pixel-wise color transformations and adaptively adjusts the sampling interval to model local non-linearities of the color transformation at high resolution. Specifically, a parameter network is first designed to generate multiple position-dependent 3-dimensional lookup tables (3D LUTs), which use the color and position of each pixel to perform pixel-wise color transformations. Then, to enhance local variations adaptively, we separate a color transform into a cascade of sub-transformations using two 3D LUTs to achieve the non-uniform sampling intervals of the color transform. Finally, a global consistent weight learning method is proposed to predict an image-level weight for each color transform, utilizing global information to enhance the overall harmony. Extensive experiments demonstrate that our AICT achieves state-of-the-art performance with a lightweight architecture. The code is available at https://github.com/aipixel/AICT .

1 Introduction

Image composition [3, 2, 24, 51] aims to combine a foreground object with a background image to create a realistic composite, which holds significant potential across various domains, including art, entertainment, commerce [3, 43, 52], and data augmentation [11, 30, 26]. However, since the foreground and background may be captured under different conditions, directly pasting the foreground onto the background usually results in an inconsistent appearance. To address this problem, image harmonization endeavors to adjust the color of the foreground to seamlessly integrate with the background, which plays a pivotal role in image editing.

Transform

Downsample

0.0

1.0

0.8

0.6

0.4

0.2

Parameter

Network

ො

𝑦(𝑖, 𝑗) = 𝐺

𝑥(𝑖, 𝑗)

𝑥

(

𝑖, 𝑗

)

ො 𝑦

(

𝑖, 𝑗

)

Traditional image harmonization methods [5, 19, 27, 28, 29, 36, 37] primarily concentrate on aligning the color statistics of the foreground to match the background using hand-crafted features. Since

these methods lack consideration for the content of the composite images, they often yield suboptimal results when there are substantial differences in appearance between the foreground and background. With the rapid advance of deep learning, learning-based methods [38, 17, 9, 8, 22, 15, 33, 6, 44, 13] have become dominant and achieved remarkable progress. These methods usually adopt encoderdecoder based structures to learn the dense pixel-to-pixel transformation between composite images and ground-truth images at a low resolution (e.g., 256 × 256 pixels), while real-world applications increasingly demand high-resolution images. Although these methods can process images with any size theoretically, the computational cost required for high-resolution images is extremely expensive.

Recently, several methods [20, 21, 46, 12] have emerged to tackle the challenge of high-resolution image harmonization. To reduce computational costs, these methods usually use a low-resolution composite image to predict transformation parameters for processing the corresponding high-resolution composite image instead of directly generating the final image. These methods can be mainly categorized into two groups. Harmonizer [20] and S 2 CRNet [21] focus on predicting image-level parameters to perform global color adjustments. However, these adjustments do not contain any semantic and local information, which leads to identical changes for pixels in different regions with the same color value. On the other hand, DCCF [46] and PCT-Net [12] predict low-resolution parameter maps and then directly upsample them to align with high-resolution composite images for pixel-wise color transformation. However, upsampling low-resolution parameter maps may introduce errors, which fails to model local non-linearities of the color transform at high resolution. In summary, these four methods ignore local color transformations across the foreground regions, which are prone to generate inharmony results in local regions.

To address this problem, we propose an Adaptive-Interval Color Transformation method (AICT) for high-resolution image harmonization. As shown in Figure 1, AICT predicts parameter maps for pixel-wise color transformations, rather than performing global color adjustments. Additionally, it adaptively adjusts the sampling interval to model local non-linearities of the color transformation at high resolution. To implement this complex transformation, we formulate the task as an image-based multiple curve estimation problem. Specifically, a parameter network is first proposed to generate multiple curves as position-dependent 3dimensional lookup tables (3D LUTs), which use the color and position of each pixel to perform pixel-wise color transformations. Then, to enhance local variations adaptively, we propose an adaptive interval learning method, which separates a color transform into a cascade of subtransformations using two position-dependent 3D LUTs to achieve the non-uniform sampling

4

6

17

19

200

240

300

340

fMSE

Harmonizer

DCCF

PCT-Net

Ours (AICT)

5

18

Better

Smaller

intervals of the color transform. Finally, a global consistent weight learning method is proposed to predict an image-level weight for each color transform, utilizing global information to enhance the overall harmony by modeling the influence of background brightness on the foreground. As shown in Figure 2, AICT achieves state-of-the-art performance in foreground-normalized MSE (fMSE) on the full-resolution images of the iHarmony4 dataset [8] while maintaining a model size comparable to PCT-Net [12]. Our main contributions are as follows:

We propose an Adaptive-Interval Color Transformation method to perform pixel-wise color transformations and model local non-linearities of the color transformation for highresolution image harmonization.

We propose an adaptive interval learning method to achieve a flexible sampling point allocation and a global consistent weight method to utilize global information to enhance the overall harmony.

Extensive experiments demonstrate that the proposed method achieves state-of-the-art performance in high-resolution image harmonization while maintaining a lightweight and simple network architecture.

2 Related Work

2.1 Image Harmonization

To reduce computational costs, existing high-resolution image harmonization methods [20, 21, 46, 12, 7, 40] usually takes a low-resolution image as input to predict transformation parameters for processing the corresponding high-resolution image instead of directly outputting the final image. For example, Harmonizer [20] employs a neural network to regress filter arguments based on lowresolution images, which are then used for several white-box filters to adjust various aspects of the foreground, including brightness, contrast, and other characteristics. S 2 CRNet [21] focuses on extracting spatial-separated embeddings from low-resolution images to predict parameters of the piece-wise curve mapping for performing color-wise transformations. Similarly, Wang et al. [40] utilize down-sampled images to predict global RGB curves for performing color correction at higher resolutions. Furthermore, they propose to predict and upsample low-resolution shading maps to address local tonal variations. These three methods predict image-level parameters to perform global color adjustments. Unlike these methods, DCCF [46] processes low-resolution images to acquire human comprehensible neural filter maps, which are subsequently upsampled and applied to the original input image. PCT-Net [12] predicts affine transformation parameter maps based on low-resolution images and upsamples them to match high-resolution images. Both methods predict low-resolution parameter maps and then upsample them directly to align with high-resolution images for pixel-wise transformations. In addition, CDTNet [7] performs pixel-to-pixel transformation at low resolution and color-to-color transformation at high resolution in parallel. It subsequently utilizes a refinement module to integrate the two intermediate outputs. The above methods typically rely on global color adjustments or the upsampling of parameter maps, which ignore local variations. Our method performs pixel-wise color transformations and models local non-linearities for high-resolution image harmonization.

2.2 LUT-based Image Enhancement

A lookup table (LUT) defines a set of tables consisting of output values, where each value can be addressed by a set of indices. When provided with an input combination, it can output a corresponding value by performing lookup and interpolation operations, so it usually serves as an effective and efficient representation of a univariate or multivariate function. Zeng et al. [49] propose to learn multiple basis 3D LUTs and predict content-dependent weights to fuse them into an image-adaptive LUT for photo enhancement. To consider global scenarios and local spatial information, Wang et al. [41] introduce a lightweight two-head weight predictor for image-level scenario adaptation and pixel-wise category fusion. Yang et al. [47] achieve a more flexible sampling point allocation in 3D LUTs by adaptively learning the non-uniform sampling intervals in the 3D color space. To enhance the expressiveness of the LUT, Yang et al. [48] decompose a single color transform into a cascade of sub-transforms and use 1D LUTs to increase cell utilization within the 3D LUT. Zhang et al. [50] employ hash techniques to reduce the space complexity of 3D LUTs. Liu et al. [23] propose to learn a context map for the pixel-level category and a group of image-adaptive coefficients for achieving context-aware 4D LUT. These methods typically predict several weight parameters to fuse pre-trained LUTs for global RGB-to-RGB transformations, while our method predicts position-dependent 3D LUTs for pixel-wise color transformations.

3 Method

Given a composite image ˜ I ∈ R H × W × 3 and a binary mask M ∈ { 0 , 1 } H × W that indicates the foreground region to be harmonized, image harmonization aims to adjust the color of the foreground region to obtain a harmonized image ˆ I ∈ R H × W × 3 close to the ground truth image I ∈ R H × W × 3 . To achieve high-resolution image harmonization at a low computational cost, we formulate the task as an image-based multiple curve estimation problem and propose an Adaptive-Interval Color Transformation method (AICT) as shown in Figure 3. AICT consists of a high resolution (HR)

Downsample

𝐶

𝐹

Lookup & Linear

Interpolation

Lookup & Linear

Interpolation

𝑖, 𝑗, 𝑥(𝑖, 𝑗)

𝑖, 𝑗

Operations

Intermediate Outputs

Combination

(𝑖, 𝑗, ො

𝑥(𝑖, 𝑗))

High Resolution Branch

Low Resolution Branch

𝑀

ሚ

𝐼

መ

𝐼

ො

𝑥(𝑖, 𝑗)

ො

𝑦(𝑖, 𝑗)

𝑥(𝑖, 𝑗)

ሚ

𝐼 low

𝑀low

Patch

Embedding

Transformer

Encoder

conv

conv

Weight Learning

Module

deconv

𝑅

{𝐰 R , 𝐰

G , 𝐰

B }

𝑇 1

𝑇2

Reconstruction

branch and a low resolution (LR) branch. In the LR branch, ˜ I and M are first downsampled to obtain a low-resolution composite image ˜ I low ∈ R H ′ × W ′ × 3 and foreground mask M low ∈ { 0 , 1 } H ′ × W ′ , where H ′ < H and W ′ < W . They are then fed into a parameter network to predict two parameter maps C ∈ R H ′ × W ′ × Q and F ∈ R H ′ × W ′ × Q and they are regarded as curves in the form of positiondependent 3D LUTs, where Q is equal to three times the number of knot points in a curve. In the HR branch, C is used to process color values and pixel coordinates to redistribute the color values of ˜ I into specific ranges of the following F for achieving pixel-wise adaptive adjustment of the sampling interval. F is used to map the redistributed color values and pixel coordinates to final color values, achieving pixel-wise color transformation. Therefore, the color transformation in our method includes two cascaded pixel-wise sub-transformations.

3.1 Adaptive Interval Learning

In the LR branch, ˜ I low and M low are fed into the parameter network to produce C and F . As shown in Figure 3, we use the Transformer encoder [13] to construct the parameter network, which consists of multiple attention layers. Firstly, the low-resolution composite image ˜ I low and foreground mask M low are divided into patches, which are then projected into the embedding space. Positional encodings are added to the embedded patches, which are processed by the Transformer encoder. The output of the Transformer encoder is reassembled to obtain the feature T 1 ∈ R 64 × 64 × 256 , and a deconvolution layer is applied to reduce the feature channel number and perform upsampling to obtain the feature T 2 ∈ R H ′ × W ′ × 12 . Subsequently, T 2 is processed through a 1 × 1 convolution to obtain the parameter map C ∈ R H ′ × W ′ × Q . We individually process each color channel and divide C into three parameter maps C R ∈ R H ′ × W ′ × K , C G ∈ R H ′ × W ′ × K , and C B ∈ R H ′ × W ′ × K , where Q is equal to 3 × K . Taking the red channel of an RGB image as an example, the corresponding parameter map is C R . To achieve adaptive interval learning, we need to reconstruct the parameter map C R . The softmax function is used to obtain the normalized interval V R ∈ [0 , 1] H ′ × W ′ × M = Softmax ( C R , axis = 3) , where axis = 3 indicates that normalization is performed along the third dimension [47] of C R . We perform cumulative summation in the third dimension of V R and then add an origin for each position to obtain the sampling coordinates K R ∈ [0 , 1] H ′ × W ′ × ( M +1) , which can be expressed as K R = [ Z ; Cumsum ( V R , axis = 3)] , where Z is a H ′ × W ′ × 1 matrix filled with zero values, and the [ · ; · ] denotes the concatenation operation. In such a way, each value in K R is within the range of 0 to 1 and maintains the monotone increasing properties along the third dimension ( K R c,b,i < K R c,b,j , for c, b ∈ I 255 0 , i, j ∈ I M 0 , and i < j ).

We use position-dependent 3D LUTs to achieve pixel-wise adaptive adjustment of the sampling interval. The position-dependent 3D LUT maps spatial coordinates and color values to new color values [35]. We treat K R as a position-dependent 3D LUT, which utilizes lookup and interpolation operations to serve as a color mapping curve. For a given coordinate ( i, j ) and its color value x ( i, j ) in the red channel of ˜ I , we can find the corresponding position ( u, v, w ) in K R , where u = i H ′ -1 H -1 , v = j W ′ -1 W -1 , w = x ( i, j ) K -1 C max and C max represents the maximum color value in the image. Based on ( u, v, w ) , we can obtain adjacent 8 sampling points in K R and perform trilinear interpolation to produce a new color value ˆ x ( i, j ) . The entire process can be formulated as

where t and l denote trilinear interpolation and lookup operations, respectively. The upper section of Figure 4 demonstrates the color redistribution process. Here, each coordinate and its corresponding color value are mapped to a new color value using K R . Similarly, we can obtain redistributed color values for the other color channels in the same way.

3.2 Pixel-Wise Color Transform

After obtaining the redistributed color values, we utilize the parameter map F to achieve pixel-wise color transformation. As shown in Figure 3, we apply a 1 × 1 convolution to the feature T 2 for obtaining the intermediate parameter map R ∈ R H ′ × W ′ × Q , which are divided into three parameter maps R R ∈ R H ′ × W ′ × K , R G ∈ R H ′ × W ′ × K , and R B ∈ R H ′ × W ′ × K according to the color channel. When inserting the foreground into a new background, the average brightness of the foreground will be influenced by the background. For example, the inserted foreground may experience overall darkening, brightening, or a bias toward a certain color. As shown in Figure 5, the foreground region un-

𝑖

𝑗

ො

𝑥(𝑖, 𝑗)

Color Redistribution

Result Prediction

𝑖

𝑗

ො

𝑦(𝑖, 𝑗)

ො

𝑥(𝑖, 𝑗)

𝑥(𝑖, 𝑗)

𝐾 R

𝐹 R

dergoes darkening after image harmonization. Therefore, we propose a globally consistent weight learning method and design a weight learning module to learn image-level parameters for controlling overall color transformations. These parameters can also be regarded as scene classification, which are adjusted based on different scenes. The weight learning module consists of two 3 × 3 convolutional layers, Batch Normalization (BN) layers [18], max-pooling layers, ReLU activation functions, and one fully-connected layer, making it lightweight. It processes T 1 to predict weight vectors w R ∈ R K , w G ∈ R K , and w B ∈ R K , which are used to multiply with R R , R G , and R B to obtain the final parameter maps F R ∈ R H ′ × W ′ × K , F G ∈ R H ′ × W ′ × K , and F B ∈ R H ′ × W ′ × K , respectively.

We also treat F R as a positiondependent 3D LUT. For a given coordinate ( i, j ) and its redistributed color value ˆ x ( i, j ) in the red channel of ˜ I , lookup and interpolation operators are performed in F R to produce the final output color value ˆ y ( i, j ) . The bottom part of Figure 4 illustrates the result prediction process. Due to the difference in resolution, each pixel in the parameter maps K R and F R corresponds to a local region of the composite image. However, the color values in a

Composite image

Ground truth

region are usually distributed within certain ranges. Therefore, K R aims to redistribute these color values across the entire color value range, thereby improving the utilization of sampling points in F R and enhancing the expressiveness of the curve. By combining these two curves ( K R and F R ), our method achieves the capability of adaptively adjusting the sampling intervals. Finally, the result images corresponding to each channel are obtained by transforming the color values at each position of ˜ I , which are then concatenated to produce the harmonized image ˆ I . As we process each color channel separately for an RGB image, the parameter network needs to predict six curves.

3.3 Loss Functions

During the training phase, the loss L high is calculated in the HR branch based on the difference between ˆ I and I . To improve the performance of AICT for images with small foregrounds, we adopt the foreground-normalized MSE loss [33], which is formulated as

where A min is a hyper-parameter to stabilize training. By replacing ˜ I with ˜ I low in the framework, we can also obtain the low-resolution harmonized image ˆ I low and the loss L low. We also apply the foreground-normalized MSE loss to minimize the difference between ˆ I low and the low-resolution version of the ground truth I low. Overall, our network can be trained by optimizing the combination of the losses above

where λ is a hyper-parameter that controls the weight of L low. Our parameter network is trained in an end-to-end manner.

4 Experiments

4.1 Experimental Setting

4.1.1 Dataset and Evaluation Metrics

Following the same settings as previous methods [20, 46, 12], we train and evaluate our method on the iHarmony4 dataset [8], which consists of four subsets (HAdobe5k, HCOCO, HDay2night, and HFlickr) and includes 73146 samples for image harmonization. Each sample contains a composite image, a corresponding foreground mask, and a ground truth image. The HAdobe5k, HCOCO, HDay2night, and HFlickr subsets consist of 2160, 4283, 133, and 828 test images, respectively. These subsets have resolutions ranging from 312 × 230 to 6048 × 4032 pixels. The width and height of images in the HCOCO, HDay2night, and HFlickr subsets are all below 1024 pixels, and only the HAdobe5k subset is composed of images with a width or height larger than 1024 pixels.

We also train and evaluate our method on the ccHarmony dataset [25], which is constructed by transferring each image across different illumination conditions to simulate natural illumination variations. This dataset contains 3080 training samples and 1180 test samples, and each sample includes a composite image, a corresponding foreground mask, and a ground truth image.

Additionally, we evaluate our method against other methods on a real composite dataset [7], which consists of 100 test samples. Each sample contains only a composite image and a corresponding foreground mask. Note that all datasets used in our experiments are publicly available.

For the quantitative performance metrics, we calculate several key indicators, including Mean Square Error (MSE), foreground Mean Square Error (fMSE), Peak Signal-to-Noise Ratio (PSNR), and Structural Similarity Index (SSIM) for each individual image in the dataset. Subsequently, we compute the average values for the entire dataset and for each specific subset. While MSE serves as a crucial evaluation metric, it tends to be biased towards images that contain larger foreground regions due to the variations in foreground sizes across the dataset. This limitation makes MSE less reliable for a comprehensive assessment of image quality. In contrast, fMSE offers a more balanced and equitable evaluation of overall quality, making it a more suitable choice for our analysis [12].

4.1.2 Implementation Details

For the iHarmony4 dataset [8], our network is trained from scratch by using Adam optimizer with β 1 = 0 . 9 , β 2 = 0 . 999 , and ϵ = e -8 . The batch size is set to 4 and the model is trained for 100 epochs. We set the learning rate as 5 e -5 for the first 50 epochs and linearly decay it to zero over the next 50 epochs. For the network design, we set K = 8 , Q = 24 , H ′ = 256 , and W ′ = 256 . For the loss function, λ is set to 0.01, and A min is set to 1000 and 100 in the HR branch and LR branch, respectively. We resize training images to ensure that the length of their

sides does not exceed 2048 pixels due to memory limitations. During the testing phase, we perform image harmonization using full-resolution images. Following [7], we also conduct tests on the HAdobe5k subset at two different resolutions ( 1024 × 1024 and 2048 × 2048 ). For the ccHarmony dataset [25], we use the parameters trained on the iHarmony4 dataset as the initial parameters, and then fine-tune our network on the training set of the ccHarmony dataset. Following [25], the test image size in this dataset is set as 256 × 256 . To augment training samples, we crop the composite image according to a random bounding box, the width and height of which are not smaller than the halved width and height of the composite image, respectively. The random horizontal flip is also applied to training samples. Our network is implemented based on the PyTorch framework and trained over approximately 75 hours on a computer equipped with two EPYC 7513 CPUs, 256GB of memory, and two 3090 GPUs.

4.2 Comparison with the State-of-the-arts

We conduct experiments and comparisons on the full-resolution test images of the iHarmony4 dataset [8]. AdaInt [47] and SepLUT [48], originally designed for image enhancement, are modified

and trained from scratch on the iHarmony4 dataset for the image harmonization task. Our AICT has three key differences compared to these methods. Firstly, they predict several weight parameters to fuse pre-trained LUTs, while our method dynamically predicts entire LUTs based on input. Secondly, they focus on global RGB-to-RGB transformations, whereas our AICT enables pixel-wise color transformations. Lastly, instead of applying adaptive interval learning and separable lookup tables for global sampling adjustments, our AICT achieves pixel-wise sampling interval adjustment to model local non-linearities in color transformation. To the best of our knowledge, only Harmonizer [20], DCCF [46], and PCT-Net [12] perform image harmonization on full-resolution images. Since Xu et al. [46] do not provide fMSE and SSIM values, we evaluate DCCF by running the provided code for comparison. Guerreiro et al. [12] propose two types of architectures: a CNN-based encoder-decoder network and a network based on a Visual Transformer (ViT) [10]. We choose the ViT-based network for comparison as it demonstrates better performance. The quantitative comparison results are shown in Table 1. We observe that AICT outperforms other methods on most metrics. Due to the limited amount of training samples in the HDay2night subset, our method achieves lower performance on this subset. To evaluate performance on low-resolution images, we also report quantitative results on the iHarmony4 dataset at a 256 × 256 resolution (see the Appendix). Additionally, we evaluate the harmonization performance on the HAdobe5k subset at a 2048 × 2048 resolution. As shown in Table 2, our method achieves the best performance across all metrics. Furthermore, we compare our method with others on this subset at a 1024 × 1024 resolution (see the Appendix).

As shown in Figure 6, we present the qualitative results of AICT against state-of-the-art methods on the full-resolution test images of the iHarmony4 dataset [8]. Compared to other methods, the images generated by AICT are closer to the ground truth images, making the composite images more realistic. More qualitative results are presented in the Appendix. Additionally, we also present the quantitative comparison results on the real composite dataset [25] to demonstrate the superiority and generalizability of our method (see the Appendix).

To further demonstrate the effectiveness of our method, we also present quantitative comparison results on the ccHarmony dataset [25] at 256 × 256 resolution. As shown in Table 3, our method also achieves the best performance across all metrics.

4.3 Ablation Study

We study the influence of the adaptive interval learning method on image harmonization performance using the iHarmony4 dataset [8], as shown in Table 4. We first remove the weight learning module and the adaptive interval learning method in AICT, denoted as 'Single'. Compared to AICT, this method achieves lower performance. We then incorporate the channel-crossing strategy [34, 35] into 'Single', denoted as 'Cross'. The parameter model uses an RGB image to predict parameter maps, which inherently include information from all color channels. Thus, the channel-crossing strategy introduces redundancy without improving performance. We also remove the adap-

tive interval learning method in AICT, denoted as 'w/o Int'. Compared to AICT, the performance of this method decreases due to the reduced expressiveness of the curves. To investigate whether using more LUTs can improve performance, we first cascade 2 and 3 LUTs for adaptive interval learning, denoted as 'Int × 2' and 'Int × 3', respectively. Then, we cascade 2 and 3 LUTs for color transformation, denoted as 'Tra × 2' and 'Tra × 3', respectively. Finally, we cascade 4 and 6 LUTs for alternating adaptive interval learning and color transformation, denoted as 'Alt × 2' and 'Alt × 3', respectively. The experimental results show that increasing the number of LUTs for adaptive interval learning does not improve performance. Furthermore, using more LUTs for color transformation decreases performance due to increased training difficulty. Additionally, we combine AdaInt [47] and SepLUT [48] for global RGB-to-RGB transformations, denoted as 'AdaInt

(c) DCCF

(d) PCT-Net

(e) AICT

(f) Ground truth

(a) Composite image

SepLUT'. Compared to AICT, one reason for the performance decline of 'AdaInt + SepLUT' is the neglect of local context.

We conduct ablation studies on the global consistent weight learning method, as shown in Table 5. First, we remove the weight vector corresponding to the R channel, denoted as 'w/o R'. Next, we remove the weight vectors for both the R and G channels, denoted as 'w/o RG'. Finally, we remove the entire weight learning module, denoted as 'w/o Weight'. As the number of removed weight vectors increases, the harmonization performance decreases, highlighting the importance of the global consistent weight learning method. We also predict spatially varying weight parameters to weight each sampling point of the parameter map R , denoted as 'Spatial'. The results for 'Spatial' indicate that learning

spatially varying weight parameters does not improve performance, as the LUT is responsible for pixel-wise color transformations. These observations demonstrate that the proposed adaptive interval learning method and global consistent weight learning method are effective.

4.4 Hyper-parameter Analyses

We conduct studies on key parameters using the iHarmony4 dataset [8], including the number of knots K , the coefficient of the LR branch loss λ , and the hyper-parameter A min in the foregroundnormalized MSE loss, as shown in Table 6. We set K to 6, 8, and 10 to analyze the influence of curves with different numbers of knots for the image harmonization task, where K is set to 8 in AICT. By comparing 'K=6' with AICT, we observe that increasing the number of knots reduces fMSE and

MSE while improving PSNR, indicating that a larger number of knots enhances the harmonization ability of the curves. However, as the number of knots increases, the performance of 'K=10' decreases, suggesting that an excessive number of knots increases network parameters. This complicates the training process of the network, making it more challenging to achieve optimal performance. To investigate the effect of the LR branch loss, we set λ to 0, 0.01, and 0.1, where λ set to 0.01 in AICT. When λ is set to 0.01, AICT achieves the best performance on all metrics, demonstrating that an optimal weight for the LR branch loss is crucial for the high-resolution image harmonization. For the foreground-normalized MSE loss, we set A min to H × W and H ′ × W ′ in the HR and LR branch, respectively, which means these objective functions are equivalent to MSE functions, denoted as 'MSE'. The results indicate that using the foreground-normalized MSE loss improves harmonization performance, as it prevents training samples with foregrounds of different sizes from being trained with varying loss magnitudes, ensuring effective training for small foreground images.

5 Conclusion

In this paper, we formulate image harmonization as an image-based multiple curve estimation problem and propose an Adaptive-Interval Color Transformation method, which predicts pixel-wise color transformation and adaptively adjusts the sampling interval to model local nonlinearities of the color transformation at high resolution. Specifically, a parameter network is first designed to generate multiple curves as position-dependent 3D LUTs, which use the color and position of each pixel to perform pixel-wise color transformation. Then, we separate a color transform into a cascade of sub-transformations

using two position-dependent 3D LUTs to achieve the non-uniform sampling intervals of the color transform. Finally, a global consistent weight learning method is proposed to predict an image-level weight for each color transform, utilizing global information to enhance the overall harmony. Experimental results demonstrate that our method achieves state-of-the-art performance in high-resolution image harmonization with a lightweight architecture.

References

Ivan Anokhin, Pavel Solovev, Denis Korzhenkov, Alexey Kharlamov, Taras Khakhulin, Aleksei Silvestrov, Sergey Nikolenko, Victor Lempitsky, and Gleb Sterkin. High-resolution daytime translation without domain labels. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 7488-7497, 2020. 14

Samaneh Azadi, Deepak Pathak, Sayna Ebrahimi, and Trevor Darrell. Compositional GAN: learning image-conditional binary composition. International Journal of Computer Vision , 128:2570-2585, 2020. 1

Bor-Chun Chen and Andrew Kae. Toward realistic image compositing with adversarial learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 8415-8424, 2019. 1

Qifeng Chen and Vladlen Koltun. Photographic image synthesis with cascaded refinement networks. In Proceedings of the IEEE International Conference on Computer Vision , pages 1511-1520, 2017. 14

Daniel Cohen-Or, Olga Sorkine, Ran Gal, Tommer Leyvand, and Ying-Qing Xu. Color harmonization. ACM Transactions on Graphics , 25(3):624-630, 2006. 1

Wenyan Cong, Li Niu, Jianfu Zhang, Jing Liang, and Liqing Zhang. Bargainnet: Background-guided domain translation for image harmonization. In IEEE International Conference on Multimedia and Expo , pages 1-6, 2021. 2, 14

Wenyan Cong, Xinhao Tao, Li Niu, Jing Liang, Xuesong Gao, Qihao Sun, and Liqing Zhang. Highresolution image harmonization via collaborative dual transformations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 18470-18479, 2022. 3, 6, 7, 8, 14, 15

Wenyan Cong, Jianfu Zhang, Li Niu, Liu Liu, Zhixin Ling, Weiyuan Li, and Liqing Zhang. Dovenet: Deep image harmonization via domain verification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 8394-8403, 2020. 2, 6, 7, 8, 10, 14, 15

Xiaodong Cun and Chi-Man Pun. Improving the harmony of the composite image by spatial-separated attention module. IEEE Transactions on Image Processing , 29:4759-4771, 2020. 2, 14

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 , 2020. 8

Debidatta Dwibedi, Ishan Misra, and Martial Hebert. Cut, paste and learn: Surprisingly easy synthesis for instance detection. In Proceedings of the IEEE International Conference on Computer Vision , pages 1301-1310, 2017. 1

Julian Jorge Andrade Guerreiro, Mitsuru Nakazawa, and Björn Stenger. Pct-net: Full resolution image harmonization using pixel-wise color transformations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 5917-5926, 2023. 2, 3, 6, 7, 8, 14, 15, 16

Zonghui Guo, Zhaorui Gu, Bing Zheng, Junyu Dong, and Haiyong Zheng. Transformer for image harmonization and beyond. IEEE Transactions on Pattern Analysis and Machine Intelligence , 45(11):1296012977, 2023. 2, 4, 14

Zonghui Guo, Dongsheng Guo, Haiyong Zheng, Zhaorui Gu, Bing Zheng, and Junyu Dong. Image harmonization with transformer. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 14870-14879, 2021. 8, 14

Zonghui Guo, Haiyong Zheng, Yufeng Jiang, Zhaorui Gu, and Bing Zheng. Intrinsic image harmonization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 1636716376, 2021. 2, 8, 14

Yucheng Hang, Bin Xia, Wenming Yang, and Qingmin Liao. SCS-Co: Self-consistent style contrastive learning for image harmonization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 19678-19687, 2022. 14

Guoqing Hao, Satoshi Iizuka, and Kazuhiro Fukui. Image harmonization with attention-based deep feature modulation. In Proceedings of the British Machine Vision Conference , volume 1, page 2, 2020. 2

Sergey Ioffe and Christian Szegedy. Batch Normalization: Accelerating deep network training by reducing internal covariate shift. In Proceedings of the International Conference on Machine Learning , volume 37, pages 448-456, 2015. 5

Jiaya Jia, Jian Sun, Chi-Keung Tang, and Heung-Yeung Shum. Drag-and-drop pasting. ACM Transactions on Graphics , 25(3):631-637, 2006. 1

Zhanghan Ke, Chunyi Sun, Lei Zhu, Ke Xu, and Rynson WH Lau. Harmonizer: Learning to perform white-box image and video harmonization. In Proceedings of the European Conference on Computer Vision , pages 690-706, 2022. 2, 3, 6, 7, 8, 14, 15, 16

Jingtang Liang, Xiaodong Cun, Chi-Man Pun, and Jue Wang. Spatial-separated curve rendering network for efficient and high-resolution image harmonization. In Proceedings of the European Conference on Computer Vision , pages 334-349, 2022. 2, 3, 14

Jun Ling, Han Xue, Li Song, Rong Xie, and Xiao Gu. Region-aware adaptive instance normalization for image harmonization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 9361-9370, 2021. 2, 8, 14

Chengxu Liu, Huan Yang, Jianlong Fu, and Xueming Qian. 4d lut: learnable context-aware 4d lookup table for image enhancement. IEEE Transactions on Image Processing , 32:4742-4756, 2023. 3

Li Niu, Wenyan Cong, Liu Liu, Yan Hong, Bo Zhang, Jing Liang, and Liqing Zhang. Making images real again: A comprehensive survey on deep image composition. arXiv preprint arXiv:2106.14490 , 2021. 1

Li Niu, Linfeng Tan, Xinhao Tao, Junyan Cao, Fengjun Guo, Teng Long, and Liqing Zhang. Deep image harmonization with globally guided feature transformation and relation distillation. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 7723-7732, 2023. 6, 7, 8, 14

Xi Ouyang, Yu Cheng, Yifan Jiang, Chun-Liang Li, and Pan Zhou. Pedestrian-synthesis-gan: Generating pedestrian data in real scene and beyond. arXiv preprint arXiv:1804.02047 , 2018. 1

Patrick Pérez, Michel Gangnet, and Andrew Blake. Poisson image editing. ACM Transactions on Graphics , 22(3):313-318, 2003. 1

Francois Pitie, Anil C Kokaram, and Rozenn Dahyot. N-dimensional probability density function transferand its application to colour transfer. In Proceedings of the IEEE International Conference on Computer Vision , pages 1434-1439, 2005. 1

Erik Reinhard, Michael Adhikhmin, Bruce Gooch, and Peter Shirley. Color transfer between images. IEEE Computer Graphics and Applications , 21(5):34-41, 2001. 1

Tal Remez, Jonathan Huang, and Matthew Brown. Learning to segment via cut-and-paste. In Proceedings of the European Conference on Computer Vision , pages 37-52, 2018. 1

Xuqian Ren and Yifan Liu. Semantic-guided multi-mask image harmonization. In Proceedings of the European Conference on Computer Vision , volume 13697, pages 564-579, 2022. 14

Xintian Shen, Jiangning Zhang, Jun Chen, Shipeng Bai, Yue Han, Yabiao Wang, Chengjie Wang, and Yong Liu. Learning global-aware kernel for image harmonization. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 7535-7544, 2023. 14

Konstantin Sofiiuk, Polina Popenova, and Anton Konushin. Foreground-aware semantic representations for image harmonization. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , pages 1620-1629, 2021. 2, 6, 7, 8, 14

Yuda Song, Hui Qian, and Xin Du. Starenhancer: Learning real-time and style-aware image enhancement. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 4126-4135, 2021. 8

Yuda Song, Hui Qian, and Xin Du. Multi-curve translator for high-resolution photorealistic image translation. In European Conference on Computer Vision , pages 126-143. Springer, 2022. 5, 8

Kalyan Sunkavalli, Micah K Johnson, Wojciech Matusik, and Hanspeter Pfister. Multi-scale image harmonization. ACM Transactions on Graphics , 29(4):1-10, 2010. 1

Michael W Tao, Micah K Johnson, and Sylvain Paris. Error-tolerant image compositing. International Journal of Computer Vision , 103(2):178-189, 2013. 1

Yi-Hsuan Tsai, Xiaohui Shen, Zhe Lin, Kalyan Sunkavalli, Xin Lu, and Ming-Hsuan Yang. Deep image harmonization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 3789-3797, 2017. 2

Jeya Maria Jose Valanarasu, He Zhang, Jianming Zhang, Yilin Wang, Zhe Lin, Jose Echevarria, Yinglan Ma, Zijun Wei, Kalyan Sunkavalli, and Vishal M Patel. Interactive portrait harmonization. In International Conference on Learning Representations , 2023. 16

Ke Wang, Michaël Gharbi, He Zhang, Zhihao Xia, and Eli Shechtman. Semi-supervised parametric real-world image harmonization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 5927-5936, 2023. 3

Tao Wang, Yong Li, Jingyang Peng, Yipeng Ma, Xian Wang, Fenglong Song, and Youliang Yan. Real-time image enhancer via learnable spatial-aware 3d lookup tables. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 2471-2480, 2021. 3

Ting-Chun Wang, Ming-Yu Liu, Jun-Yan Zhu, Andrew Tao, Jan Kautz, and Bryan Catanzaro. Highresolution image synthesis and semantic manipulation with conditional gans. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 8798-8807, 2018. 14

Shuchen Weng, Wenbo Li, Dawei Li, Hongxia Jin, and Boxin Shi. MISC: multi-condition injection and spatially-adaptive compositing for conditional person image synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 7741-7749, 2020. 1

Yazhou Xing, Yu Li, Xintao Wang, Ye Zhu, and Qifeng Chen. Composite photograph harmonization with complete background cues. In Proceedings of the ACM International Conference on Multimedia , pages 2296-2304, 2022. 2

Ke Xu, Gerhard Petrus Hancke, and Rynson WH Lau. Learning image harmonization in the linear color space. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 12570-12579, 2023. 14

Ben Xue, Shenghui Ran, Quan Chen, Rongfei Jia, Binqiang Zhao, and Xing Tang. DCCF: deep comprehensible color filter learning framework for high-resolution image harmonization. In Proceedings of the European Conference on Computer Vision , pages 300-316. Springer, 2022. 2, 3, 6, 7, 8, 14, 15, 16

Canqian Yang, Meiguang Jin, Xu Jia, Yi Xu, and Ying Chen. Adaint: Learning adaptive intervals for 3d lookup tables on real-time image enhancement. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 17522-17531, 2022. 3, 4, 7, 8, 14, 15

Canqian Yang, Meiguang Jin, Yi Xu, Rui Zhang, Ying Chen, and Huaida Liu. Seplut: Separable imageadaptive lookup tables for real-time image enhancement. In Proceedings of the European Conference on Computer Vision , volume 13678, pages 201-217, 2022. 3, 7, 8, 14, 15

Hui Zeng, Jianrui Cai, Lida Li, Zisheng Cao, and Lei Zhang. Learning image-adaptive 3d lookup tables for high performance photo enhancement in real-time. IEEE Transactions on Pattern Analysis and Machine Intelligence , 44(4):2058-2073, 2020. 3

Fengyi Zhang, Lin Zhang, Tianjun Zhang, and Dongqing Wang. Adaptively hashing 3dluts for lightweight real-time image enhancement. In IEEE International Conference on Multimedia and Expo , pages 27712776. IEEE, 2023. 3

Shengping Zhang, Quanling Meng, Qinglin Liu, Liqiang Nie, Bineng Zhong, Xiaopeng Fan, and Rongrong Ji. Interactive object placement with reinforcement learning. In Proceedings of the 40th International Conference on Machine Learning , pages 41510-41522, 2023. 1

Song-Hai Zhang, Zheng-Ping Zhou, Bin Liu, Xi Dong, and Peter Hall. What and where: A context-based recommendation system for object insertion. Computational Visual Media , 6(1):79-93, 2020. 1

A Appendix

A.1 More Quantitative Results

In this section, we provide supplementary quantitative results to further demonstrate the advantages of our proposed method. First, we evaluate AICT against more state-of-the-art methods using lowresolution test images from the iHarmony4 dataset [8]. Following previous work, we downsample these images to 256 × 256 pixels to obtain low-resolution versions. It is important to note that AICT employs parameters trained on high-resolution images from the iHarmony4 dataset [8] and uses low-resolution images as input to perform pixel-wise color transformation instead of downsampling high-resolution output images for comparison. As shown in Table 7, AICT achieves superior performance across most metrics, which demonstrates the effectiveness of our method on lowresolution images. However, our method performs slightly worse on the HDay2night subset due to the limited number of training images in this subset.

We also compare AICT with stateof-the-art methods on the HAdobe5k subset at a resolution of 1024 × 1024 . As shown in Table 8, our method achieves the best performance in terms of MSE and PSNR, and performs slightly worse than CDTNet256 [7] in terms of fMSE and SSIM. It is important to note that CDTNet [7] requires retraining for images of varying resolutions. In contrast, AICT demonstrates flexibility by using parameters trained on high-resolution images from the iHarmony4 dataset [8] to effectively process images at various resolutions.

In order to compare AICT with stateof-the-art methods at ultra-high resolutions, we collect two ultra highresolution benchmarks (over4K and over5K) by selecting images with res-

olutions exceeding 4K ( 4096 × 2160 ) and 5K ( 5120 × 2880 ) from the HAdobe5k subset. As shown in Table 9, our method achieves the best performance on the over4K benchmark in terms of fMSE, MSE, and SSIM, and the second-best performance in terms of SSIM. As shown in Table 10, on the over5K benchmark, our method achieves the best performance in terms of MSE and PSNR and the second-best performance in terms of fMSE and SSIM.

We further compare the FLOPs, memory cost, and inference time of our method with state-of-the-art methods on the HAdobe5k subset. These metrics are crucial for evaluating the computational efficiency and scalability of the models. Our tests were conducted on a computer equipped with 32GB of memory and a GeForce GTX 1070 Ti GPU. Tables 11 and 12 show the comparison results at 1024 × 1024 and 2048 × 2048 resolutions, respectively.

At both resolutions, Harmonizer [20] has a significant advantage in terms of FLOPs, while DCCF [46] has the lowest memory cost. At a resolution of 1024 × 1024 , our method is comparable to PCTNet [12] in FLOPs, memory cost, and inference time. However, at a resolution of 2048 × 2048 , our method demonstrates lower memory cost and shorter inference time compared to PCT-Net. One important reason is that the parameter maps in our method have a fixed size, whereas PCT-Net needs to align the parameter maps with the composite images, resulting in higher memory consumption and longer inference time.

A.2 More Qualitative Results

We present additional qualitative comparison results on the iHarmony4 dataset [8], shown in Figure 7. Specifically, we compare our proposed method with Harmonizer [20], DCCF [46], and PCT-Net [12]. The qualitative results demonstrate that our method achieves superior harmonization performance, generating images that closely resemble the ground truth. Our method excels in adjusting the overall brightness and preserving

local details in the foreground, leading to more realistic and visually pleasing composite images.

A.3 Evaluation on Real Composite Images

Weprovide qualitative results on a real composite dataset [7], which presents a significant challenge by combining foreground regions from different light fields with background regions. The resolution of this dataset ranges from 1024 × 1024 to 6016 × 4000 pixels. As shown in Figure 8, AICT outperforms other methods in producing visually superior results in real-world scenarios. The images generated by AICT show a more harmonious appearance, with both overall brightness and local foreground

(c) DCCF

(d) PCT-Net

(e) AICT

(f) Ground truth

(a) Composite image

details appearing more realistic. Following [39], we first randomly select 20 high-resolution real composite images from this dataset to conduct a user study. Then, 20 volunteers independently rank

(c) DCCF

(d) PCT-Net

(e) AICT

(a) Composite image

the predictions from 1 to 3 based on visual quality. Scores of 3, 2, and 1 are assigned for ranks 1, 2, and 3, respectively. The mean scores for each method are presented in the table below. As shown in Table 13, our method achieves the highest score. The above experimental results demonstrate the superiority and generalizability of our method.

A.4 Broader Impacts

This paper proposes a pixel-wise high-resolution image harmonization method aimed at adjusting the color of the foreground to seamlessly integrate with the background, thereby enhancing the realism

of composite images and making a significant contribution to the image composition community. Simultaneously, our research holds paramount importance in domains such as art, entertainment, and commerce. However, it is worth noting that image harmonization techniques could potentially be exploited to create deceptive or misleading visual content. To address this concern, there is a pressing need for the development of image manipulation detection methods. Additionally, we emphasize the importance of raising awareness about the capabilities and limitations of such methods to effectively mitigate their adverse impacts.

A.5 Limitations

A primary limitation of our method lies in its incapability to effectively address foreground elements like mirrors and glass. This is because the model cannot accurately simulate the interplay between light and various materials in such scenarios. Figure 9 illustrates some failure cases where our method struggles to harmonize images containing such elements.

(a) Composite image

(b) AICT

(c) Ground truth

(a) Composite image

(b) AICT

(c) Ground truth

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The main claims made in the abstract and introduction accurately reflect the paper's contributions and scope.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: The paper discusses the limitations of the work performed by the authors in Section A.5.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [NA]

Justification: The paper does not include theoretical results.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: The paper fully discloses all the information needed to reproduce the main experimental results of the paper in Section 3.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: The data used are publicly available, and the code can be found at https: //github.com/aipixel/AICT , along with sufficient instructions to faithfully reproduce the main experimental results.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: The paper specifies all the training and test details in Section 4.1.2.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [No]

Justification: The code can be found at https://github.com/aipixel/AICT . The results can be reproduced using the provided code and the implementation details described in the paper, so no error bars are reported in this paper.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: The paper provides sufficient information on the computer resources in Section 4.1.2.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: The research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: The paper discusses both potential positive societal impacts and negative societal impacts of the work performed in Section A.4.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: The paper poses no such risks.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: All assets used in the paper, including code, data, and models, have been properly credited to their original creators. The licenses are explicitly mentioned in Section 4.1.1 and 4.2.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [Yes]

Justification: The paper proposes a new model, and the documentation is provided in Section 3.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: The paper does not involve crowdsourcing or research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: The paper does not involve crowdsourcing or research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work proposes the Adaptive-Interval Color Transformation (AICT) method for harmonizing high-resolution composite images. The proposed AICT first uses a parameter network to predict multiple curves (as 3D LUTs) to perform pixel-wise color transformation at low-resolution. The AICT then adjusts the sampling intervals of the color transformation by splitting each predicted 3D LUT into two 3D LUTs. Last, AICT uses a global consistent weight learning method to predict an image-level weight for each color transform. Experiments on the iHarmony4 [8] and ccHarmony [26] datasets show promising quantitative results of the proposed AICT.\n\nSTRENGTHS:\nOriginality: \n(+) The strength is that the difference to existing HR image harmonization methods is clear.\n\nQuality: \n(+) The proposed method is complete and technically sound. (+) The experiment settings (evaluation datasets, metrics, and competing methods) are convincing.\n\nClarity: \n(+) The paper presents sufficient implementation details for reproducing the proposed method.\n\nSignificance: \n(+) Quantitative results in Tables 1-3, and 6-7 show superior or comparable performance of AICT in harmonizing HR and LR composite images.\n\nWEAKNESSES:\nOriginality: \nThis work proposes the AICT method to address an existing task of harmonizing high-resolution (HR) composite images. While previous HR image harmonization methods (e.g., [12,46]) predict and then upsample low-resolution (LR) parameter maps for harmonization, the proposed AICT also predicts LR parameter maps but applies two 3D LUTs to harmonize the colors in a coarse-to-fine manner. (-) One weakness is the limited/vague novelty of using two 3D LUTs for color transformation, which is a combination of ideas from existing LUT-based image enhancement methods [47, 48] but lacks necessary discussions and experimental verifications.\n\nQuality: \n(-) The paper mentions the ``non-linearities of the color transform at high resolution’’ many times as the key problem suffered by existing methods and addressed by the proposed AICT. However, the paper does not explain what the ``non-linearities’’ are, nor do the visual results in Figures 7, 8, and 9 illustrate such advantages.\n(-) The paper claims AICT being lightweight and computationally efficient but Figure 2 only demonstrates that AICT has a small model size. According to Tables 8 and 9, AICT runs much slower than [21] and [46] and is comparable to [12]. Hence, such a claim needs to be tuned down.\n(-) The paper claims the ``global consistent weight method’’ to be novel but the paper does not explain its novelty.\n\nClarity: \nThere are a few weak points regarding the paper writing. \n(-) The paper mentions the ``adaptive interval learning method’’ and the ``global consistent weight method’’ as the second main contribution (line 86-87, page 2). However, the Method section discusses the proposed ``global consistent weight method’’ very briefly as a part of subsection 3.2 (line 207-214, page 5). The authors are suggested to either explain the ``global consistent weight method’’ in detail regarding its novelty or remove the corresponding claim in the second contribution.\n(-) According to Figure 3 and Section 3 (line 134-144, page 4) the LR branch of the proposed AICT outputs two parameter maps (C and F). However, Eq. (3) indicates that there is a low-resolution harmonized image predicted by the LR branch.\n(-) Figure 1 is not self-contained. The S2CRNet [22] is missing in Figure 2.\n(-) The sub-title of Section 2.1 is better: ``High-resolution Image Harmonization.\n(-) There are no discussions on the relations between the proposed AICT and existing methods in Section 2.1 and 2.2.\n(-) There are many repeated sentences in, for example,(Line 5-6, Line 60-63, Line 374-378), and (Line 7-13, Line 67-78).\n(-) The references contain redundant items, such as [13,14].\n(-) ``M=6’’ and ``M=10’’ should be ``K=6’’ and ``K=10’’.\n(-) It is better to move the Tables 6 and 7 to the main paper.\n\nSignificance: \n(-) The visual results in Figure 7-9 do not show significant improvements.\n(-) The main quantitative performance gain seems to come from the foreground-normalized MSE loss [34], according to Tables 4 and 5.\nThe paper presents a harmonization method that can produce better quantitative results, but the visual advantages illustrated in this paper are not significant\n\nQUESTIONS:\nQ1: What is the novelty of the proposed 3D LUTs-based adaptive-interval color transformation, compared to the non-uniform sampling interval learning in Yang et al. [47] and the decomposition of a single color transform into two sub-transformations in Yang et al. [48]. It is suggested to explain the challenges of directly combining [47] and [48], and how the proposed AICT addresses such challenges. Comparison results between AICT and a simple combination of [47] and [48] are suggested to provide.\nQ2: Can authors report and explain the results of incorporating more 3D LUTs?\nQ3: According to Tables 4 and 5, the foreground-normalized MSE loss [34] is more important to the final results compared to the ``w/o Weight’’ and ``w/o Color’’. Can authors explain this and/or provide more results for clarification?\nQ4: Any more ablations on the proposed ``global weight’’?\nQ5: Can authors report non-reference image quality metrics (e.g., NIQE and LPIPS) for the harmonized real composite images in Figure 9?\nQ6: Can authors provide intermediate visual results of the ablated models?\nQ7: Can authors explain why using the channel-crossing strategy does not improve the performance?\nQ8: Any reason not to use SSIM as the evaluation metric?\n\nLIMITATIONS:\nThe paper has discussed the limitations and potential negative societal impact. I cannot think of any further critical points.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-06-13"}

comments: SUMMARY:
This paper proposes an AdaptiveInterval Color Transformation method (AICT) for high-resolution image harmonization, which predicts pixel-wise color transformation and adaptively adjusts the sampling interval to model local non-linearities of the color transformation at high resolution.

STRENGTHS:
1. The paper is overall well-written and easy to understand.
2. The idea is neat and clean.
3. The experimental results look good.

WEAKNESSES:
1. The authors should conduct experiments using the training/test set of HAdobe5k which focuses on high-resolution, and report the results under different resolutions (1024, 2048, etc). More baselines should be compared in this setting.
2. The authors should test on high-resolution real composite images and conduct user study.
3. The authors are suggested to evaluate the method on ccHarmony dataset, which can reflect the illumination variation more faithfully.
4. The authors should discuss the limitation of the proposed method and show some failure cases.
5. More efficiency analyses (FLOPs, memory, time) should be provided.

QUESTIONS:
Please address the questions in "Weakness".

LIMITATIONS:
See the "Weakness".
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-06-16

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a new method for harmonizing the color of foreground objects added to scenes with the colors of the background original image. The authors focus on a model able to present good results on high-resolution images. The idea is to learn two different sets of Look-Up-Tables. The first set aims at modifying the input values to re-scale them in a way that the second set can perform pixel-wise editing. The authors show the ability of their method in three different datasets.\n\nSTRENGTHS:\n- The general idea of using two LUT-like transformations is interesting.\n- Results outperform the state of the art.\n\nWEAKNESSES:\n- The paper is difficult to follow at the beginning, as the authors abuse notation regarding 3DLUTs in color image processing. In color image processing 3DLUTs are a function that goes from R^3 to R^3, i.e. given an input (R,G,B) values they output a (R',G',B') value ---see for example [1]---. Therefore, in color image processing 3DLUTs cannot be approximated by curves (contrary to what is said by the authors in lines 138-139). What the authors are doing is learning spatial-aware 1D LUTs, i.e. a function from R^3 to R that given an input (x,y,R) outputs a value R'. It is true that some papers (citation 24 on the paper) already abuse notation when speaking of 4DLUTs. However, in their case, they are not redefining an already existing term, as it is 3DLUTs. Therefore, the authors of this manuscript need to rewrite different parts of the paper refraining from using the term 3DLUT to define their approach. As it currently stands, it cannot be accepted due to this conceptual error.\n\n- Metrics are insufficient. MSE and PSNR are strongly correlated metrics. In the end, PSNR=10log_{10}(MAX^2/MSE), where MAX is the maximum possible value. Therefore, the authors need to add further metrics such as LPIPS, DeltaE as perceptually-based full-referenced metrics, and NIQE as a non-reference one.\n\n- The paper will be of much larger interest if the authors present the results of a user study, for example as it was done in PCT-Net (reference 12 in this paper) or in [2]. \n\n- Figures 1,3, and 4 are versions of the same general idea of Figure. It will be beneficial for the paper if the authors can present just 1 Figure in which all the small differences are included.\n\n[1] Conde, M. V., et al. Nilut: Conditional neural implicit 3d lookup tables for image enhancement, AAAI 2024\n[2] Valanarasu, J. M. J., et al. (2022). Interactive portrait harmonization, ICLR 2023\n\nQUESTIONS:\nFirst, please address the weaknesses mentioned before.\n\nAlso, when reading this paper, I got the feeling that: \"This idea is mostly AdaInt + Mask\". I may have lost some important component here, so please guide me through why my feeling may not be correct.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a new method called Adaptive-Interval Color Transformation (AICT) for high-resolution image harmonization. The key ideas are:\n\n1. Predicting pixel-wise color transformations using a parameter network that generates multiple 3D lookup tables (LUTs).\n2. Separating the color transform into cascaded sub-transformations using two 3D LUTs to adaptively adjust the sampling intervals and model local non-linearities.\n3. Using a global consistent weight learning module to predict image-level weights for each transformation to enhance overall harmony.\n\nExtensive experiments show that AICT achieves state-of-the-art performance on benchmark datasets while being computationally efficient.\n\nSTRENGTHS:\n1. **Novel approach to high-resolution image harmonization**: The paper proposes a new method, Adaptive-Interval Color Transformation (AICT), which addresses the challenging problem of harmonizing high-resolution composite images. The key idea of using adaptive sampling intervals in the color transformation is a novel contribution that sets AICT apart from prior works.\n\n2. **Improved quantitative performance on benchmark datasets**: AICT achieves state-of-the-art results on the widely-used iHarmony4 and HCOCO datasets, outperforming previous methods in terms of fMSE, MSE, and PSNR metrics. These quantitative improvements, although relatively small in some cases, demonstrate the effectiveness of the proposed technical innovations.\n\n3. **Computational efficiency and scalability**: The paper presents a detailed analysis of the computational complexity and memory usage of AICT, showing that it achieves a favorable trade-off between performance and efficiency compared to existing high-resolution harmonization methods. The scalability of AICT to higher resolutions (e.g., 2048x2048) is also demonstrated, which is important for practical applications.\n\n4. **Insights into the role of adaptive sampling intervals**: Through ablation studies and visualizations, the paper provides some insights into the importance of adaptive sampling intervals in the color transformation process. The comparison between fixed and adaptive intervals highlights the benefits of the proposed approach in terms of handling local color variations and edge artifacts.\n\nWEAKNESSES:\n## Unrealistic Settings\nThe input resolutions in the main experiments (1024x1024 and 2048x2048) are still relatively low compared to professional compositing workflows, which often deal with 4K or higher resolutions. The effectiveness of AICT for ultra high-resolution images is unclear and should be validated with appropriate benchmarks.\n\n## Lack of Comparison to Extensively Studied Prior Art: Proposed Approach Appears to be a Trivial Amalgamation of Established Techniques\n- The use of 3D LUTs for color transformations is not a fundamentally new idea in image processing. Many previous works, especially in the photo enhancement domain, have used similar techniques. The paper does not sufficiently explain how AICT's specific LUT-based approach is distinct from prior methods.\n- Adaptively adjusting the sampling intervals of color transformations has also been explored in works like AdaInt. While AICT's dual cascaded LUTs are a nice extension, this still seems like an incremental contribution building on existing ideas.\n- The global weight learning module likewise appears very similar to the global adjustment parameters used in existing works. More discussion is needed on how AICT's weights differ and what additional capabilities they enable.\n\n## Missing motivations for method modules\n\n- The global weight learning module seems disconnected from the rest of the method. It's unclear why weighting the predicted LUTs based on global image features would improve harmonization quality. The authors should explain the reasoning behind this design choice and how it relates to the adaptive interval learning.\n- The global weight learning module predicts a single scalar weight for each LUT, which is then uniformly applied to all pixels. This global approach contradicts the motivation of AICT to enable local color adjustments. A more spatially-varying weighting scheme would be more consistent with the overall goal of the method.\n\n\n## Evaluation\n\n- The use of cascaded dual 3D LUTs is a key aspect of AICT, but is not ablated. Testing a single LUT baseline, more numbers of LUTs or alternative compositions of LUTs would help validate the necessity of the proposed decomposition. \n- The user study results are not reported, making it difficult to assess AICT's perceptual quality compared to other methods. Quantitative metrics like fMSE and PSNR do not always correlate well with human judgments, so user ratings would provide valuable additional evidence for AICT's benefits.\n\n## Performance\nWhile the method outperforms previous approaches, the gains on some metrics (e.g. PSNR) are relatively modest (<1 dB). It would be good to see more analysis on what types of images/scenarios benefit most from AICT.\n\n\nOverall, the reviewer acknowledges that this paper addresses an important topic using generally reasonable methods. The initial score will be increased if the rebuttal effectively addresses the concerns mentioned above.\n\nQUESTIONS:\nPlease see weakness.\n\nLIMITATIONS:\nPlease see weakness.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-13"}