P-BiC: Ultra-High-Definition Image Demoireing via Patch Bilateral Compensation

Authors: zeyu  xiao
xinchao  wang
Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-06 16:34:29

Abstract: People are nowadays using smartphones to capture photos from multimedia platfroms. The presence of moir\'e patterns resulting from spectral aliasing can significantly degrade the visual quality of images, particularly in ultra-high-definition (UHD) images. However, existing demoir\'eing methods have mostly been designed for low-definition images, making them unsuitable for handling moir\'e patterns in UHD images due to their substantial memory requirements. In this paper, we propose a novel patch bilateral compensation network (P-BiC) for the demoir\'e pattern removal in UHD images, which is memory-efficient and prior-knowledge-based. Specifically, we divide the UHD images into small patches and perform patch-level demoir\'eing to maintain the low memory cost even for ultra-large image sizes. Moreover, a pivotal insight, namely that the green channel of an image remains relatively less affected by moir\'e patterns, while the tone information in moir\'e images is still well-retained despite color shifts, is directly harnessed for the purpose of bilateral compensation. The bilateral compensation is achieved by two key components in our P-BiC, i.e., a green-guided detail transfer (G$^2$DT) module that complements distorted features with the intact content, and a style-aware tone adjustment (STA) module for the color adjustment. We quantitatively and qualitatively evaluate the effectiveness of P-BiC with extensive experiments.

Main paper: 
 introduction: 
 Moiré patterns often arise when a camera captures a subject with a patterned texture or a repetitive structure that has a similar frequency to the camera's sensor, resulting in visual distortion and a loss of detail in the captured image. To eliminate moiré patterns and restore high-quality images, image demoiréing has been proposed and drawn increasing attention in both academia and industry. Recent works have made remarkable progress in improving the performance of image demoiréing by introducing advanced neural network architectures [6, 7, 14, 16, 19-21, 24, 26, 34, 39-42, 44, 46, 52, 53, 55] . However, those methods are primarily designed for removing the moiré patterns in low-definition (LD) images, often resulting in intense computational cost and poor performance due to more complex and pronounced moiré patterns when applied to high-definition (HD) images, e.g., photographs acquired by present smartphones.Recent works have been proposed to eliminate moiré patterns in HD and even ultra-HD (UHD) images [12, 18] , which are becoming more popular in current imaging systems. Compared with LD images, the moiré patterns in HD and UHD images are more complicated because of the severe interference between the much finer patterns, such as the individual pixels on an HD display and the sampling grid of image sensors [7] . In addition to the complexity of moiré patterns in HD and UHD images, an HD/UHD demoiréing method also needs to consider the large amount of memory cost with the increasing image size. FHDe 2 Net [7] is a pioneer work for the removal of moiré patterns in HD images, in which the moiré patterns are first erased at the low-resolution stage, and the textures are then refined at the high-resolution stage. However, FHDe 2 Net generates artifacts when performing demoiréing for UHD images, which are more susceptible to moiré patterns due to the higher spatial frequency. In contrast, ESDNet [46] has been proposed recently for the demoiréing in UHD images with promising performance, where a semantic-aligned scale-aware module is designed for the scale variation of moiré patterns. Despite the high performance, two limitations exist in ESDNet: (i) it neglects the internal moiré-specific properties, leading to sub-optimal results with unrealistic artifacts and incorrect tones (see Figure 1 Missing data points indicate that moiré images at these resolutions cannot be reconstructed on an 80GB GPU (NVIDIA A100) due to memory limitations. P-BiC stands out by not only restoring moiré images up to a resolution of 7168 × 7168, but also achieving a remarkable 4-fold reduction in memory utilization when restoring an image of 3072 × 3072, as compared to the method presented in [46] . Notably, these values are derived from feeding the entire image into the GPU for demoiréing.directly input the entire image without cropping into the network, the issue of "out of memory" arises, particularly when processing images of higher resolutions. For instance, on a high-performance GPU like the NVIDIA A100 with 80GB of memory, the ESDNet method can handle a maximum resolution of 3072 × 3072 before encountering this limitation. To address the above problems, in this paper, we introduce a novel method named the Patch Bilateral Compensation Network (P-BiC) for UHD image demoiréing. This novel method offers memory efficiency and leverages prior knowledge to guide the process. The key to memory efficiency lies in partitioning a full-resolution feature map into several smaller patches, a technique that enables processing UHD moiré images within the constraints of limited memory resources. To illustrate, P-BiC can conduct demoiréing on a 7168 × 7168 resolution image while staying within the memory limits of a single NVIDIA A100 GPU, as depicted in Figure 1 (right).Furthermore, we leverage a fundamental observation as the guiding principle for our bilateral compensation approach. Specifically, we exploit the fact that the green channel of an image is less affected by moiré patterns, which can be attributed to the red and blue channels having half the sampling frequency of the green channel in the color filter array [1, 6] . This inherent property allows the moiré image to retain well-preserved color information despite the presence of moiré artifacts. Building upon this observation, we disentangle the UHD demoiréing task into two distinct sub-tasks, which are facilitated by the core components of P-BiC: the Green-Guided Detail Transfer (G 2 DT) module and the Style-Aware Tone Adjustment (STA) module. The G 2 DT module effectively employs the features of the green channel to complement distorted features with intact content information. Conversely, the STA module undertakes color representation adjustments for the features of the green channels, which might otherwise lose significant tone details. The synergistic integration of these components empowers P-BiC to deliver both efficient GPU memory usage and high-quality demoiréing outcomes, as showcased in Figure 1 . These achievements stem from the design of a patch-level processing strategy and the strategic utilization of prior observations to guide bilateral compensation.We summarize our contributions as follows. (1) We identify the limitations of existing UHD image demoiréing methods, i.e., the intense memory cost, and the poor-quality results. Inspired by that, a novel network is specifically designed in this work for UHD image demoiréing, achieving both memory efficiency and higher quality results. (2) We propose a novel patch bilateral compensation network (P-BiC) that operates at the patch level to restore UHD moiré images with limited memory use and utilizes a prior observation for bilateral compensation. In particular, two key modules of P-BiC, namely the G 2 DT module and the STA module, serve for the bilateral compensation. (3) We validate the effectiveness of P-BiC through comprehensive experiments on benchmark datasets. It significantly outperforms existing methods while maintaining lower computational costs, thus highlighting the efficacy and practicality of our proposed method. 
related work: 
 Conventional image demoiréing. To suppress moiré patterns, Kim et al. [9] place an optical low-pass filter in front of the lens to avoid aliasing. However, it cannot eliminate moiré artifacts while reserving the image details. Similarly, it is time-consuming to capture a moiré-free image by selecting an optimal angle of lens [10, 22] . Later methods have relied on various filtering or image decomposition techniques. Wei et al. [33] propose a median-Gaussian filtering method for eliminating moiré patterns in X-ray microscopy images. Liu et al. [13] utilize a low-rank and sparse matrix decomposition-based method to remove moiré patterns from texture images. Yang et al. [42] utilize layer decomposition on polyphase components for demoiréing. Deep image demoiréing. Sun et al. [26] propose a multi-scale demoiréing network with the first benchmark dataset that captures real LCD screens for training and evaluating demoiréing models. He et al. [6] add annotations of different types of moiré After extracting green-channel image 𝐺 𝑀 from 𝐼 𝑀 , both of them are converted into the feature domain, denoted as 𝐹 𝑀 and 𝐹 𝐺 . These rescaled features are then divided into patches, followed by the bilateral compensation, which consists of the G 2 DT module and the STA module. The outputs of both modules are then fused and assembled to 𝐹 . Finally, 𝐹 is fed to the feature decoder for moiré-free image reconstruction.patterns to the dataset in [26] and propose MopNet for demoiréing. Yue et al. [48] propose a multiplicative operation-based network that simultaneously removes moiré patterns and improves image brightness. Zheng et al. [52, 53] and He et al. [7] further exploit DCT domain priors for demoiréing, while Liu et al. [16] utilize wavelet domain features to separate the frequencies of moiré patterns. Liu et al. [17] introduce an additional input of a focuseddefocused image pair for demoiréing with a self-supervision scheme. Wang et al. [29] propose a coarse-to-fine disentangling framework for demoiréing. Zhang et al. [51] propose a patch-based framework for efficient demoiréing based on existing methods. Recently, Yu et al. [46] propose ESDNet, achieving promising performance on image demoiréing. High-definition image restoration. With the rapid advancement of mobile devices, modern smartphones are now capable of capturing HD and UHD images, underscoring the importance of research in HD and UHD image restoration for practical applications. For instance, Zheng et al. [54] introduce a dehazing method tailored for UHD images, employing a multi-guided bilateral learning framework. This approach integrates both global and local guidance information to produce more accurate and visually appealing dehazed images. Deng et al. [4] propose a multi-scale separable network designed for UHD video deblurring, harnessing both spatial and temporal cues to generate sharp and clear video frames. Yi et al. [43] introduce a contextual residual aggregation mechanism by learning the change of image resolution for UHD image inpainting. Feng et al. [5] propose GLSGN for UHD image restoration. The GLSGN adopts both local and global pathways to restore images in a step-wise manner, and is effective in deraining, dehazing, and reflection removal. In the realm of image demoiréing, FHDe 2 Net [7] and ESDNet [46] have been developed to address moiré patterns in HD and UHD images. Green channel prior. As discussed in [28] , the CMOS sensor has different sensitivity to light of different wavelengths or colors, and in most illumination conditions, green channels are brighter than red and blue channels in Bayer pattern CFA images. As a result, the green channel has more texture information than red/blue channels in most natural images. While previous works, such as MopNet [6] , have acknowledged that the green channel is less affected by moiré interference, they have not fully exploited this inherent characteristic. Some existing methods [8, 15, 27, 49] utilize the green channel to reconstruct the other color channels. However, they do not delve into the specific details of the green channel or explore its complementary nature with RGB images. In contrast to these approaches, our motivation stems from the observation that the green channel is less impacted by moiré but lacks color information. Additionally, the tone information in moiré images remains largely intact despite color shifts. 
method 3.1 overview: 
 As shown in Figure 2 , given a moiré-degraded image 𝐼 𝑀 ∈ R 3×𝐻×𝑊 , P-BiC can generate a moiré-free image Î , which should be close to the ground-truth image 𝐼 𝐺𝑇 . To fully exploit the internal moiréspecific property, we extract the green-channel moiré image 𝐺 𝑀 from 𝐼 𝑀 , which is less affected by moiré patterns. We first feed 𝐼 𝑀 and 𝐺 𝑀 to pre-processing heads to enlarge the receptive field [46] , followed by the feature encoders to extract multi-scale features{𝐹 𝑀 1 , 𝐹 𝑀 2 , 𝐹 𝑀 3 } and {𝐹 𝐺 1 , 𝐹 𝐺 2 , 𝐹 𝐺 3 }.The sizes of 𝐹 𝑀 3 and 𝐹 𝐺 3 are 𝑐×ℎ×𝑤. The encoders consist of three building blocks (convolutional layers and residual blocks), and the second and third blocks halve the size of the feature maps with stride 2. Both encoders do not share weights. To save the memory costs, we divide 𝐹 𝑀 G 2 DT module and the STA module for bilateral compensation. Next, the output feature patches from the G 2 DT module and the STA module are fused and assembled into a high-resolution feature 𝐹 , which shares the same size as 𝐹 𝑀 3 and 𝐹 𝐺 3 . Finally, 𝐹 is fed to the feature decoder, followed by the pixel-shuffle upsampling operation, to obtain the reconstructed moiré-free image Î . The feature encoders and the feature decoder are connected via skip-connections, allowing features containing rich high-resolution information to facilitate the reconstruction of moiré-free images.Image demoiréing methods face significant challenges when dealing with UHD images due to high memory requirements, which can limit their practicality and scalability. We propose a memoryefficient strategy for UHD image demoiréing that centers on reducing the resolution of the processed feature 𝐹 . To this end, we present four possible methods, each with multiple design choices, for achieving memory saving in UHD demoiréing. Cropping non-overlapped patches. Given a high-resolution feature, the reflect padding operation is first adopted, and we then crop 𝐹 into multiple non-overlapping patches {𝐹 𝑖 |𝑖 = 1, 2, . . . , 𝑁 𝑛𝑜 }. Each divided patch share the same size𝐹 𝑖 ∈ R 𝑐 × (ℎ/ √ 𝑁 𝑛𝑜 ) × (𝑊 / √ 𝑁 𝑛𝑜 ). After we process each divided patch separately with the bilateral compensation, we combine them in their original order. Cropping overlapped patches. We first utilize the reflect padding operation, and we then crop 𝐹 into multiple overlapped patches {𝐹 𝑖 |𝑖 = 1, 2, . . . , 𝑁 𝑜 } using a 𝐾 × 𝐾 pixels sliding window with a stride of 𝑆. After the loop of patch-level bilateral compensation, we discard the overlapping regions on the processed patches and assemble them into a complete feature. Unlearnable rescaling. We feed a pooled small-sized feature instead of feeding the full-resolution one to the bilateral compensation part. We then upsample the processed feature to the original size. Learnable rescaling. We utilize convolutional layers with different strides for learnable rescaling, and we feed the compact feature to the bilateral compensation stage.Further experiments demonstrate that cropping non-overlapped patches with 𝑁 𝑛𝑜 = 4 is an effective strategy that can achieve a favorable balance between performance and computational costs.The presence of moiré patterns introduces a challenging task of demoiréing, as they exhibit a wide-ranging frequency spectrum that intertwines with the underlying images. This complexity is further amplified when dealing with UHD images due to heightened interference. Consequently, it becomes imperative to fully harness the intrinsic moiré-specific characteristics to achieve effective UHD demoiréing. A fundamental insight emerges from the structure of a typical Bayer color filter array: the green channel's sampling frequency is twice that of the red and blue channels. This property results in the green channel being less affected by moiré patterns, with the tone information in the moiré image remaining relatively intact. Our proposed P-BiC capitalizes on this observation by introducing the G 2 DT and STA modules. These modules operate at the patch level and engage in bilateral compensation to exploit the moiré-specific properties, ultimately contributing to improved UHD demoiréing results. Green-guided detail transfer. As shown in Figure 4 , the G 2 DT module is designed to transfer the details contained in green-channel images to the moiré images. We first concatenate the moiré image feature 𝐹 𝑀 3,𝑖 and the moiré feature 𝐼 𝑀↓ 3,𝑖 , followed by a convolutional layer to generate the enhanced moiré feature 𝑭 𝑀 𝑖 . The enhanced green image feature 𝑭 𝐺 3,𝑖 can be obtained in the same wayEQUATIONwhere [•, •] denotes the concatenate operation. Such a process combines the complementary advantages of the feature domain and the image domain, which promote the enrichment of feature representation [25] . 𝑭 𝑀 3,𝑖 and 𝑭 𝐺 3,𝑖 are then fed to the simple yet effective green-channel branch and the moiré branch. The green-channel branch aims to refine and enhance the details of the green image feature. It first convolves the green channel feature 𝑭 𝐺 3,𝑖 , and the residual 𝑅 𝐺 3,𝑖 between the convolved green-channel feature and the moiré feature 𝑭 𝑀 𝑖 is then convolved asEQUATIONSimilarly, the details in the moiré image feature are refined and enhanced asEQUATIONThe outputs of both branches are then concatenated and aggregated, obtaining 𝑭 ′ 3,𝑖 . In practice, however, we find feed 𝑭 ′ 3,𝑖 to the following part in P-BiC tends to generate artifacts since the G 2 DT module is performed at the patch level, ignoring global information contained in high-resolution input. We therefore feed the rescaled green-channel image 𝐼 𝐺 ↓ with 𝑭 ′ 3,𝑖 to the fusion operation for further aggregation. Please refer to the supplementary for the details of the fusion operation. Style-aware tone adjustment. The green-channel image and the moiré image share the same content and texture, but their color distributions differ. To address this, we introduce the STA module, which aims to map the distribution of the green-channel image feature to that of the moiré image feature for tone adjustment.As shown in Figure 5 , we first obtain the enhanced green image feature 𝑭 𝐺 done in the G 2 DT module. We then obtain the style feature F 𝑀 3,𝑖 viaEQUATIONwhere ResB(•) denotes the residual block. The green channel feature and the style feature are initially concatenated before being fed into a convolutional layer, resulting in the generation of two parameters, denoted as 𝛼 and 𝛽. Both 𝛼 and 𝛽 possess the same dimensions as the style feature. Then, instance normalization is applied to the green-channel feature asEQUATIONwhere 𝜇 𝐺,𝑐 3,𝑖 and 𝜎 𝐺,𝑐 3,𝑖 are the mean and standard deviation of 𝑭 𝐺 3,𝑖 in channel 𝑐. We then update 𝛼 and 𝛽 with the mean and standard deviation of the style feature asEQUATIONFinally, 𝛼 and 𝛽 are multiplied and added to the normalized green channel feature in an element-wise manner asEQUATIONThis process ensures a style-aware adjustment of the tone of the green channel feature.The STA module draws inspiration from [23] . However, it differentiates itself from [23] . While [23] utilizes segmentation maps to generate two parameters, the convolutions within our STA module accept both the green channel image/feature and the style feature as inputs, enabling them to learn the differences between them. Furthermore, after obtaining the parameters 𝛽 and 𝛾 from these convolutions, we combine them with the mean and standard deviation of the green channel feature. Such design contributes to the style-aware adjustment of the green channel feature.Upon obtaining the processed patch features, 𝑭 𝐺 2 𝐷𝑇 3,𝑖 and 𝑭 𝑆𝑇 𝐴 3,𝑖 are combined through concatenation, followed by a convolutional layer. These aggregated features are then fed into the feature decoder for further reconstruction. It is noteworthy that, akin to ESDNet [46] , P-BiC generates three hierarchical moiré-free predictions Î 3, Î 2, Î , with Î 3 and Î 2 being employed and supervised during the training phase. Reconstruction loss. We adopt 𝐿 1 loss as the reconstruction loss to supervise hierarchical moiré-free predictions asEQUATIONwhere 𝐼 𝐺𝑇 𝑖 is the rescaled ground-truth image, which has the same size as Î𝑖 . Perceptual loss. The perceptual loss is expressed asEQUATIONwhere 𝜙 𝑗 (•) denotes the 𝑗-th layer of the pretrained VGG16 network. Here, we use conv3_3 (after ReLU). Full objective. Our full objective is defined asEQUATION)where 𝜆 is the weighting factor to balance two loss terms. 
experiments 4.1 experimental setting: 
 Datasets. We conduct experiments on four public image demoiréing datasets: TIP2018 [26] , LCDMoiré [47] , FHDMi [7] , and UHDM [46] . Evaluation metrics. For our evaluation, we employ widely-accepted image quality assessment metrics, including PSNR, SSIM [32] , and LPIPS [50] . Past studies have indicated that LPIPS offers greater consistency with human perception, making it particularly suitable for assessing demoiréing results [7, 46] . It is crucial to emphasize that in line with established conventions in the field, existing methods have typically used only PSNR and SSIM metrics on the TIP2018 and LCDmoiré datasets. In order to ensure fairness and comparability, we follow this convention in our evaluations. Implementation details. We implement our algorithm using Py-Torch on a single NVIDIA RTX 3090 GPU. We utilize the Adam optimizer with parameters 𝛽 1 = 0.9 and 𝛽 2 = 0.999. The learning rate is initially set to 0.0002 and scheduled by cyclic cosine annealing. We set 𝜆 = 1 to balance different loss terms. Our P-BiC is trained with different configurations depending on different datasets. For TIP2018, we train for 100 epochs with a batch size of 4. For FHDMi and LCDMoiré, we train for 200 epochs with a batch size of 2. Similarly, for UHDM, we train for 200 epochs with a batch size of 2.We conduct a comprehensive comparison between our P-BiC and several state-of-the-art image demoiréing methods that have publicly available source code. These methods include DMCNN [26] , MDDM [2] , WDNet [16] , MopNet [6] , MBCNN [52] , FHDe 2 Net [7] , Wang [29] , ESDNet, and its larger variant ESDNet-L [46] . It is worth noting that DDA [51] is designed specifically for low-resolution moiré images and aims to enhance the performance and efficiency Table 2 : Computational cost comparisons measured on an NVIDIA A100 GPU between P-BiC and UHD image demoiréing methods on UHDM. In our evaluations, we feed the entire UHD moiré image as the input. The PSNR results of ESDNet and ESDNet-L are extracted from [46] . The unit of runtime is second, and memory here denotes the peak memory (GB). of existing networks. Therefore, we do not include it in the comparison, as its design philosophy is not directly applicable to the UHD image demoiréing task addressed in this paper. Quantitative comparison. Table 1 provides a quantitative comparison across four benchmark testsets, highlighting the consistently superior performance of P-BiC. Notably, P-BiC demonstrates significant outperformance compared to the state-of-the-art UHD image demoiréing method, ESDNet-L. Specifically, P-BiC achieves improvements of 0.45dB, 0.21dB, 0.57dB, and 0.88dB in terms of PSNR on the TIP2018, LCDmoiré, FHDMi, and UHDM datasets, respectively. An important observation is that P-BiC attains these remarkable results while utilizing only 4.922M parameters, in stark contrast to ESDNet-L's utilization of 10.623M parameters. This emphasizes P-BiC's efficiency in UHD image demoiréing. The computational costs are detailed in Table 2 . It is worth noting that, for our comparison, we input the entire UHD moiré image into the network for inference. While existing methods can employ patch-wise inference followed by the merging operation, this approach often leads to artifacts at the borders and suboptimal results. Additionally, border pixels may not fully benefit from neighboring pixels outside the patch for image restoration [3] . To address these issues, we perform a comprehensive evaluation by conducting inference on the entire image. Conclusively, P-BiC distinctly surpasses ESDNet in terms of both performance and computational efficiency. Another illustration of computational efficiency can be observed in Figure 1 . It is evident that our P-BiC is capable of operating at higher resolutions, whereas ESDNet and ESDNet-L struggle in this aspect. Qualitative comparison. Exemplar visual results for various methods are presented in Figure 1 and Figure 6 . In Figure 1 , it is notable that only P-BiC is able to generate color-accurate results, whereas other methods fail to reconstruct the colors accurately. Shifting our focus to Figure 6 , the comparison becomes even more compelling. In this context, it becomes exceedingly clear that P-BiC consistently generates results of a superior nature, characterized by enhanced details and precise color rendering. This, in turn, contributes to the generation of more perceptually pleasing and visually captivating moiré-free results.Memory-efficient UHD image demoiréing. Table 3 presents a comprehensive comparison of four distinct patch-dividing strategies, all geared towards achieving memory-efficient UHD demoiréing. These strategies encompass non-overlapping patch cropping with varying quantities, overlapping patch cropping characterized by different window sizes (𝐾) and strides (𝑆), unlearnable methodologies grounded in maxpooling and avgpooling operations, and the learnable approach which relies on convolutional layers with diverse strides. It is evident from the results that the unlearnable strategy yields the least favorable reconstruction outcomes. This is attributed to the inherent limitations of utilizing pooling operations for direct resolution reduction, which results in irreversible information loss and consequently, suboptimal results. In contrast, the strategy centered around cropping patches exhibits improved performance. Additionally, a trade-off between the number of non-overlapping patches and reconstruction performance becomes apparent. While increasing the count of cropped patches aids in curbing memory usage, it simultaneously elongates runtime and dampens PSNR values. This stems from the fact that an upsurge in the number of cropped patches leads to a decrease in the resolution of individual patches, thereby constraining the extent of global information available for moiré pattern removal. Given these findings, we opt for the non-overlapping strategy, setting the number of cropped patches at 4. This choice ensures a judicious balance between performance and computational efficiency, corroborated by empirical evidence. Effectiveness of bilateral compensation. Bilateral compensation, leveraging the intrinsic moiré-specific characteristics, is a key component of our approach. To showcase its effectiveness, we design and analyze several variants: ( (3) STA-w/o-Adjustment: here, we remove the style-aware tone adjustment operation from the original STA module. The quantitative results are presented in Table 6 . Notably, a significant performance drop is observed when the tone adjustment operation is removed. Specifically, on TIP2018, FHDMi, and UHDM datasets, the performance decreases by 0.36dB, 0.22dB, and 0.09 dB, underlining the critical importance of the tone adjustment operation. Effectiveness of the green channel. As analyzed earlier, the green channel contains crucial information that can be effectively utilized for UHD demoiréing. To visually illustrate this, we visualize the red, green, blue, and grayscale versions of both moiré-distorted and clear images in Figure 7 . The visualization indeed confirms that the green channel is less impacted by moiré patterns compared to the other channels.Despite the promising performance showcased above, P-BiC does have certain limitations that warrant further investigation. In realworld scenarios, especially when image texture and moiré patterns are intricately intertwined, P-BiC might encounter challenges in effectively discriminating between these elements. Please refer to the supplementary material.In light of these limitations, our future research direction will encompass various areas to enhance P-BiC's capabilities and applicability: (1) Real-time processing:. One crucial avenue for future research involves bridging the gap towards real-time performance. To achieve this, we plan to design more efficient and lightweight architectures tailored for real-time UHD demoiré processing. This will contribute to making P-BiC a more practical solution for applications requiring instant processing. (1) Advanced patch cropping. While our exploration into patch cropping strategies is valuable, future work can delve into more sophisticated techniques, such as irregular patch cropping. These methods can adapt to diverse moiré pattern shapes and distributions, enhancing the versatility of P-BiC across a wider range of scenarios. (3) Extension to other tasks. Beyond UHD demoiréing, we intend to extend the utility of P-BiC to other UHD image reconstruction tasks [11, 30, 31, 45] . (4) Video demoiréing. Given the success of P-BiC in UHD image demoiréing, an exciting avenue involves its expansion to video demoiréing tasks [35] [36] [37] [38] . Extending P-BiC's capabilities to handle video sequences with moiréing patterns would further amplify its practical relevance and applicability. 
conclusion: 
 In this paper, we address the limitations of existing UHD image demoiréing methods by proposing a novel patch bilateral compensation network (P-BiC) that achieves both the memory efficiency and the high-quality UHD image demoiréing. Specifically, P-BiC performs UHD moiré pattern removal at the patch level to maintain the low memory cost, and leverages an internal moiré-specific property to enable bilateral compensation via two key modules, namely the G 2 DT module and the STA module. The G 2 DT module supplements moiré-distorted features with the intact content and the STA module performs style-aware tone adjustment for color correction. P-BiC achieves state-of-the-art performance on diverse datasets, surpassing previous methods by significant margins. Moreover, P-BiC achieves these results while maintaining low computational costs, making it a practical solution for UHD moiré pattern removal. 
 
 Appendix:  


Decision: This submission has been reviewed by three experts, the final ratings from whom are all positive after rebuttal (although one of them held a negative vote in initial phase, after rebuttal, the reviewer raised his/her score). Considering the quality of the submission, the comments from the reviewers, and the rebuttal from the authors, the paper can be accepted by the conference.

Reviews: review: paper_summary: paper_summary: This manuscript proposes an image moire removal method for UHD images, namely P-BiC. P-BiC divides the image into smaller patches and processes them individually, thus reducing memory usage. It also leverages the observation that the green channel of an image is less affected by moire patterns than the red and blue channels. This allows P-BiC to use the green channel as a guide to restoring the other channels, effectively removing the moire patterns while preserving image details and color accuracy. The method has been evaluated on benchmark datasets and has shown superior performance compared to existing methods.
strength_weakness: strength_weakness: + This paper is well-motivated. The demoiring on UHD images does pose a challenge to existing methods.
+ The quantitative and qualitative results show its superiority over existing methods.
+ The manuscript is in good structure and is easy to follow.
limitations: limitations: + Tab.1, There seems to be a typo in SSIM for Wang et al. on TIP2018
+ Although claimed to handle 7000x7000 images, the authors don't present results under that scenario. I don't consider this as a serious limitation.
paper_summary: paper_summary: This work focuses on addressing the problem of ultra-high-definition image demoireing. Specifically, the authors propose a memory-efficient method called P-BiC for removing moire patterns in ultra-high-definition (UHD) images. Specifically, by dividing UHD images into patches, P-BiC performs patch-level demoireing, reducing memory requirements. In addition, leveraging insights about the green channel's resilience to moire patterns, P-BiC guides detail transfer and tone adjustment for effective bilateral compensation. The experimental results show the effectiveness of the proposed method.
strength_weakness: strength_weakness: 1. The paper is overall well-written and easy to follow.
2. The proposed method achieves better performance than existing methods.
3. This work provides extensive ablation study experiments, which have verified the effectiveness of the proposed method.
limitations: limitations: 1. The paper lacks a comparison of results on real-life examples for UHD demoireing, which is essential for evaluating the method's performance in practical scenarios.
2. Unclear motivation and lack of insight. The authors claim that the green channel of an image remains relatively less affected by moiré patterns. However, there is no evidence to verify this phenomenon. 
3. Some related UHD restoration methods, such as LLFormer [a], UHDFour [b], and NSEN [c], are not discussed in the related work section. [a]. Wang et al., Ultra-high-definition low-light image enhancement: A benchmark and
transformer-based method. [b]. Li et al., Embedding fourier for ultra-high-definition low-light image enhancement. [c]. Yu et al., Learning Non-Uniform-Sampling for Ultra-High-Definition Image Enhancement.
paper_summary: paper_summary: This paper presents a supervised method for the demoireing task. To enhance memory efficiency, training was conducted using small patches (P-BiC). By utilizing the relatively abundant information in the green channel, the method successfully controls details (by G^2DT) and color tone (by STA), thereby improving reconstruction performance.
strength_weakness: strength_weakness: - The method utilizes the characteristics of the green channel in RAW data (e.g., Bayer CFA) for demoireing and provides a thorough analysis related to this approach.
- In terms of fidelity, the method outperforms existing methods.
limitations: limitations: - Although four types of patch utilization are presented, the main contribution seems to be achieving memory efficiency by applying simple non-overlapped patches without introducing any special patch cropping techniques. Feeding image patches instead of the entire image is a well-known approach for efficiency, so it is unclear how this can be considered a novelty. The improvement in peak memory shown in Table 2 does not seem significant, and the runtime nearly doubles, which is critical for real-time applications. To emphasize efficiency, it would have been better to present other metrics such as total memory usage, FLOPS, and BOPS.
- In Table 1, LPIPS is measured in only two tables. It would have been more reliable for assessing perceptual quality related to color tone if LPIPS had been measured in the other tables as well. Additionally, using FID scores could be another method for evaluation.