Recaptured Raw Screen Image and Video Demoiréing via Channel and Spatial Modulations
Authors: Huanjing Yue
Yijia Cheng
Xin Liu
Jingyu Yang
Venue: NeurIPS 2023 poster (2023)
Keywords: Raw image demoiréing
raw video demoiréing
video demoiréing dataset

Abstract: Capturing screen contents by smartphone cameras has become a common way for information sharing. However, these images and videos are often degraded by moiré patterns, which are caused by frequency aliasing between the camera filter array and digital display grids. We observe that the moiré patterns in raw domain is simpler than those in sRGB domain, and the moiré patterns in raw color channels have different properties. Therefore, we propose an image and video demoiréing network tailored for raw inputs. We introduce a color-separated feature branch, and it is fused with the traditional feature-mixed branch via channel and spatial modulations. Specifically, the channel modulation utilizes modulated color-separated features to enhance the color-mixed features. The spatial modulation utilizes the feature with large receptive field to modulate the feature with small receptive field. In addition, we build the first well-aligned raw video demoiréing (RawVDemoiré) dataset and propose an efficient temporal alignment method by inserting alternating patterns. Experiments demonstrate that our method achieves state-of-the-art performance for both image and video demoiréing. Our dataset and code will be released after the acceptance of this work.

Recaptured Raw Screen Image and Video Demoiréing via Channel and Spatial Modulations
Huanjing Yue
Yijia Cheng Tianjin University
Tianjin University Tianjin, China
Tianjin, China
huanjing. yueetju edu.u.uuedduee edu.
yijia_ chengetju. edu. edu. cn
Xin Liu
Jingyu Yang
Tianjin University
Tianjin University Tianjin, China yjyotju. edu. cn
Tianjin, China
Lappeenranta-Late University of Technology LUT
Lappeenranta, Finland Linuxsino@gmail. com
Abstract
Capturing screen contents by smartphone cameras has become common way for information sharing. However, these images and videos are often degraded by moiré patterns, which are caused by frequency aliasing between the camera filter array and digital display grids. We observe that the moiré patterns in raw domain is simpler than those in sRGB domain, and the moiré patterns in raw color channels have different properties. Therefore, we propose an image and video demoiréing network tailored for raw inputs. We introduce a color-separated feature branch, and it is fused with the traditional feature-mint branch via channel and spatial modulations. Specifically, the channel modulation utilizes modulated color-separated features enhance the color-mixed features. The spatial modulation utilizes the feature with large receptive field to modulate the feature with small receptive field. In addition, we build the first well-aligned raw video demoiréing (Raw VDmmoiré) dataset and propose efficient temporal alignment method by inserting alternating patterns. Experiments demonstrate that our method achieves state-of te-thhe-td art performance for both image and video demoriéing. We have released the code and dataset in https://jituubbbcom//uuueeennn/p
Introduction
With the popularity of mobile phone cameras, the way of information transmission has gradually evolved from text to images and videos. However, when capture the contents displayed on digital screens, the recaptured images and videos are heavily influenced by the moiré patterns (i.c., colored stripes, as shown Fig. 1) due to the frequency aliasing, and the visual contrast are also degraded. Therefore, the task of image and video demoiréing is emerging.
In recent years, image demoiréing has attained growing attention and the demoiréing performance has been greatly improved with the development of deep learning networks and the constructed paired demoiréing datasets. Since the moiré patterns are various scales and is distinguishable in frequency domain, they are removed by constructing multi-scale fusion networks [19; 6; 28; 14], frequency
Corresponding author. This work was supported in part by the National Natural Science Foundation of China under Grant 62072331, Grant 62231018, and Grant 62171309.
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

Mouré Full images
SRGB
R-channel
G1-channel
G2 chameel
B B-charzel
Visualization of moiré patterns in different color channels and their corresponding DCT spectra. For the sRGB channel, we only present the DCT spectrum of its intensity (Y) channel.
Figure 1:
domain modeling [1 41; 42], or moiré pattern classification [5]. These works all perform demoiréing in standard RGB (sRGB) domain. In these works, the moiré patterns are not only caused by frequency aliasing between the camera's 's color filter array (CFA) and the grids of display screen, but also by the demosaicing (interpolating new pixels, further aggravating frequency aliasing) algorithms in the image signal processing (ISP) [34]. In contrast, the moiré patterns in raw domain is simpler, the raw data contains the most original information, and processing moiré patterns in raw domain is more consistent with the imaging pipeline. Therefore, Yue et al. proposed demoiriéing [34] and demonstrated the superiority of the raw domain processing over sRGB domain processing. However, they directly processed the raw input with an encoder-decoder network, without exploring the special properties of moiré patterns in raw domain. Therefore, developing tailored demoiréing method for raw inputs is demanded.
raw domain
Compared with image demoiréing, video demoiréing is rarely studied. Recently, Dai et al. [3] proposed the first video demoireing network with a temporal consistency loss and constructed a video demoiréing dataset. However, their dataset is constructed with one camera and one screen, which limits the diversity of moiré patterns. In addition, there are no raw videos this dataset. Meanwhile, we observe that raw video processing is emerging in other low-level vision tasks, such raw video enhancement [1], denoising [31; 23], and super-resolution [13; 37J. Therefore, developing a raw video demoiréing method is also demanded.
We observe that the moiré patterns in different color channels of the raw inputs have different properties. Fig. presents the moiré patterns from two frames. For each image, the top row visualizes the moire patterns in different color channels the zoomed patches and the bottom row presents the DCT (Discrete Cosine Transform) spectra of corresponding channels. The patches are normalized for better observation. It can be observed that the moiré patterns in sRGB domain have complex colors and the scales are different. Meanwhile, the moiré patterns in its raw RGGB channels have different intensities and stripes. For example, for the first image, the moiré patterns B-channel are more heavier while for the second image, the R-channel has more moiré patterns. This also leads to different DCT spectra, where the bright spots represent the frequencies of moiré patterns (more analysis is provided in the appendix). However, this phenomena does not exist in the sRGB domain since the demosaicing operation mixes these moiré patterns and the green channel always has the least moiré pattern due to large sampling rate. Therefore, how utilize the above properties for raw image (video) demoiréing is important. In this work, we utilize different weights to modulate the color-separated features from the four color channels. In addition, the period of moiré patterns may be large. Therefore, the features with a large receptive field can modulate the features with a small receptive field. Based the above observations, propose perform raw image and video demoiréing via channel and spatial modulations. Our contributions are as follows.
First, considering that the traditional cross-channel convolution will enhance the shared moiré patterns in different channels, we propose to utilize two branches. One branch cross-channel convolution for shared feature enhance and the other is color- color-group group convolution with learnable parameters modulating

---

different color groups. Fusing the two branch features together can help remove moiré patterns and recover shared visual structures.
Second, the moiré patterns usually have a large spatial period. Therefore, we propose to utilize features with large receptive field (realized by depth-wise convolution) modulate the features with small receptive field. In this way, we can use small computation cost to take advantage of the correlations in large receptive field.
Third, we construct the first real raw video demoiréing dataset with temporal and spatial aligned pairs. Since the frame rate of raw video capturing is unstable, propose efficient temporal alignment method by inserting alternating patterss. Experimental results demonstrate that our method achieves the best results on both raw video and image demoriéing datasets.
2 Related Works
2.1 Image and Video Demoiréing
Compared with traditional demoriéing methods [17: 18; 26; 27], the convolutional neural network (CNN) based image demoiréing methods have achieved much better performance, especially for the recaptured screen image demoriéing. Since the moiré stripes usually have different scales, many methods propose to fuse the features from different scales together [19; 6; 28; 14] to erase both large scale and small scale moiré stripes. Meanwhile, some methods propose to model the moiré patterns in frequency domain, such as waveler domain filtering [11] and DCT domain filtering [41; 42]. Wang et al. combined the two strategies together by developing a coarse-to-fin dmoiréing network. There also methods trying use the classification priors. For example, MopNet [5] removed moiré patterns according to the classification of moiré stripes while Yue et al. proposed to remove moire patterns according to the content classification priors [34]. Liu et al. proposed utilize unfocused image to help demoriéing of focused image [12]. Many demoiréing methods also emerged in famous competitions, such as the AIM challenge [30] in ICCV 2019 and NTIRE challenge [29] in CVPR 2020. Real-time demoriéing has also been explored [40]. With the development of transformer, a U-Shaped transformer was proposed remove moiré patterns [22]. Besides supervised demoireing, unsupervised demoireing is also explored [33]. All these methods are proposed for sRGB image demoiréing except [34], which verified that raw domain demoiréing is more appealing.
Compared with image demoiréing, video demoiréing was rarely studied. Dai et al. proposed a video demoiréing method with relation-basee temporal consistency loss [3]. They also constructed moiré video dataset for supervised learning. However, this dataset only contains one camera and one screen, which limits the diversity of moiré patterns.
Different from them, we propose a raw image and video demoiréing method by taking advantage of the moiré differences among different color channels in raw domain.
2.2 Raw Image and Video Processing
Since the raw data is linear with imaging intensity and contains the most original information with wider bit depth, many lowelevel vision tasks have been explored in raw domain. The representative works include raw image and video denoising [10; 21; 31; 23; 32; 4], raw image and video super- resolution [39; 13; 37], raw image and video enhancement [2: 1], and raw image and video HDR [43; 36], etc. These works have demonstrated that raw domain processing can generate better results than sRGB domain processing. For demoiréing, the raw domain processing is lagging behind, especially for raw video demoiréing. In addition, the existed raw demoiréing method [34] did not tailor the networks according to the properties of raw inputs. In this work, we propose to solve image and video demoiréing in raw domain by utilizing the moiré differences raw domain.
2.3 Demoiréing Datasets
The demoiréing performance heavily depends on the training set. Therefore, many datasets for moiré removal have been proposed, which have become essential resources for researchers to train and evaluate their algorithms. The datasets in sRGB domain include synthetic datasets, i.e., L.Dmoire [30] and CFAmoire [29], and real datasets, such as TIP2018 [19], FHDMi [6] and MRBI [35]. Raw
3

---

Display 10fps
Displa 10ips
Recapture 30pps
Recapture 30fps
30pps
<300ps
(a) Idoel situation
(b) Actull situation
Figure 2: Visualization of the temporal confusion problem during recapturing. (a) is the ideal recapturing situation. (b) shows the actual situation, where the video displaying and recapturing do not have the same start timestamps and the raw domain recording has unstable frame rates. The temporal confusion frames are marked with red boxes.
image demoiréing dataset has also been proposed [34]. During dataset construction, they usually design sophisticated methods to make the recaptured image and the source image be spatial aligned.
Compared with image dataset construction, it is since it is difficult to keep the displaying and recording be synchronized, which may lead to multi- frame confusion. Dai et al. constructed the first video demoiréing dataset [3] but there many confused frames in their dataset. addtion, there is no raw videos in this dataset. Therefore, in this work, we construct the first raw video demoiréing dataset by inserting alternating patterns help select temporal aligned frames from the source and recaptured videos.
difficult construct video demoiréing dataset
3 Raw Video Demoiréing Dataset Construction
Although there many image demoiréing datasets, there's few video demoiréing dataset. In this work, we construct the first raw video demoiréing dataset by capturing the screen with hand- held cameras. We observe that directly capturing the videos displayed on the screen cannot keep synchronization between the displayed frame and recaptured frame due to two problems. First, we cannot exactly make the recording and displaying start timestamps be the same. Second, the camera recording frame rates and video displaying frame rates cannot be exactly matched. Especially for raw video capturing, the recording frame rates are not stable due the storage speed limits. For example, when we set the recording frame rate to 30 fps, the real recording frame rate varies from 27 to 30 fps. The two problems both lead multi-frame confusion, as shown in Fig. 2 (b).
To solve the first problem, similar to [3], we insert several white frames at the start and the end of source videos for determining the start and the end of the video. For the second problem, need to insert a flag on the source frames to help distinguish the confused frames and unconfused frames. naive flag is expanding the source frames with white black surrounding areas. In this way, we can select the unconfused frames based on the pixel intensity in the surrounding areas. However, this will cause unconsistent brightness of selected frames. Therefore, we propose insert slash and backslash patterns alternatively surrounding the source frames, as shown in Fig. 3 (a). As shown in Fig. 3 (b), for confused frames, the patterns surrounding the frame will become crossed patterns since displayed frames switch but they are recorded into one frame. Therefore, we can select the unconfused frames based on the patterns, which is formulated as flag recognition step. Specifically, to reduce the influence of noise and moiré artifacts on pattern recognition, we first perform Gaussion filtering on the original captured frames (Vaw). then we improve the contrast of the filtering results via sigmoid function to make the edge clear. Afterwards, we use Canny edge detection extract edges on the surrounding area and utilize Hough transform to detect the endpoints of the lines, and calculate the average slope of all lines. If the mean slope value is around (-1), it corresponds to the slash (backslash) pattern. Otherwise, if the slope value is around 0, corresponds to the cross pattern. In this way, can remove the confused frames with cross pattern. Then we remove the repetitive frames with the same content by calculating the difference of slope, resulting in the selected raw video Ve..
The capturing process, as shown in Fig. 3, can be summarized as follows. Given a of source frames, we first expand them by alternatively inserting slash and backslash patterns in the surrounding areas. Then, we insert white frames at the beginning and ending of the source frames. For these white frames, we insert a box to help indicate the recording regions in the following frames. The reorganized frames, denoted as TARB are displayed on the screen with frame rate of 10 fps. Then, recapture these frames with mobile phones. Since need record the raw frames, utilize

---

the MotionCam 2 App to help save and export raw frames. The recording frame rate is set to 30 fps. Note that we need fix the exposure parameters and white balance parameters when shooting. otherwise the color and brightness of recaptured videos will be unstable. The recaptured video frames are denoted as Veaw Then we select the suitable recaptured frames via deleting the white frames, recognizing the flag pattern, and removing repetitive frames, resulting in Viaw In this way, we construct the temporal consistent pairs (TRGB, Vnn)).
Note that although TRGB and Vaw are temporal aligned, they have spatial displacements. Therefore, for each video pair, we further utilize spatial alignment strategies to make them pixel-wise aligned. Similar to [34], we first utilize homography transform to coarsely align T with V where t is the frame index. Then, we utilize DeepFlow [24] perform pixel-wise alignment, resulting in RG Finally, we crop the center area of TRGB, and Vaw generating the video pairs with resolution of 720p. Note that, we also export the sRGB version of the recaptured videos, denoted as VARGB- Therefore, we also generate the video demoiréing pairs in sRGB domain, namely TAGGB and VERGB-
The displayed source videos are from DAVIS [15], Vimeo*, high-quality cartoons, and screen recordings, such as scrolling documents, webpages, and GUIs. To ensure the quality of the videos, we select the video whose resolution is ( larger than) 1080p for recapturing. In this way, our dataset includes natural scenes, documents, GUIs, webpages, etc. We totally generated 300 video pairs and each video contains 60 frames. To ensure the diversity of moiré patterns. we utilize four different combinations of display screens and mobile phones. We change the distances between our camera and the display screen from 10 to 25 centimeters. The view angle changes from to 30 degrees. Please refer to the appendix for more details about our dataset.
Compared with the video demoiréing dataset in [3], our dataset contains more diverse contents, and the moiré patterns are also more diverse than that in [3]. In addition, our dataset has no confusion frames while [3] contains confusion frames since its selecting strategy will fail when the frame rate is unstable. We also provide comparisons with synthesized datasets in the appendix. In summary, our dataset contains taw-GGB and RGB-RGB pairs, which can facilitate moiré removal in both raw and sRGB domains.
crplating
Source
Figure 3: The pipeline of video preprocessing and recapturing.
4 Approach
4.1 Network Structure Overview
Fig. 4 presents the proposed video and image demoiréing network. For videos, in order to utilize the neighboring frame information, the input is three consecutive frames (Vitw Vt Viw: v/t1) and the output the sRGB result of the central frame (RR))) The whole pipeline includes preprocessing. feature extraction, alignment, temporal fusion, and reconstruction modules. For the preprocessing, we pack the raw Bayer input with size of H W into four color channels with size of H/2 W/2 x Then, we utilize the normal convolution and group convolution (GConv) to generate color-mixed features and separ-sep features (i.e., the features for the R, GI, G2 and B channels are generated separately). Following EDVR [20] and VDmoiré [3], we utilize pyramid cascading deformable (PCD) alignment [20] to align the features of V(-1 and Vt+1 with those of V The color-mixed feature and solor-separated feature share the same alignment offsets. Then, these features are temporal fused together via a temporal fusion module at multiple scales, which realized by weighted sum of features from different frames [3]. The fusion weights are predicted from the concatenated temporal features. Afterwards, the fused features go through the reconstruction module to reconstruct the
Phtps//www.maatnncaapp..cmm hhtps//vvimooccm//
5

---

demoiréing result at multiple scales. Following [3; 25], the features at different scales are densely connected make them communicate with each other. Note that, all the operations are applied on two different kinds of features. As shown in the top row of Fig. 4, the first branch is constructed by color-separated features, and the bottom three branches are constructed by color-mixed features. The two kinds of features are fused together at multiple scales with the proposed channel and spatial modulations, which are described in the following. Note that, for image demoiréing, we directly remove the PCD alignment and temporal fusion modules. The remaining modules construct our raw image demoiréing network.
Timpumi
Pack
Pack
lignment
jurra
Pack
Fasion
Preooceesingg and festur estaction
Recoretructinn
Colorssreetted Feeture
Color-mizet Feeture
Cony Blocks
GCom Blocks
CSMod
PiselStrftle
Figure 4: The proposed raw video demoiréing network. The input three consecutive raw frames (Vawl, Viw, Vis and the output is the multi-scale result of the central frame in sRGB domain, and
the first scale result O SRGB is the demoiréing result. The downsampling (4) is realized by strided convolution and upsampling (f) is realized by bilinear interpolation and convolution. For image demoiréing, the PCD alignment and temporal fusion modules removed.
4.2 Channel and Spatial Modulations
In the literature, for image demoiréing, we generally utilize the traditional convolutions to convolve with channels, generating RGB-mixed (or RGGB-mixed for raw inputs) features. This operator can help recover the shared structures of the three channels. However, this also enhances the shared moiré patterns existed in the three channels. For raw images, we observe that the moiré patterns of the four channels in domain are different. As shown in Fig. 1, the moiré intensities in the four channels are different and usually exists one channel with fewer moiré patterns. If we only utilize traditional convolution to recover shared structures, the moiré patterns will also spread to all channels. Therefore, besides the traditional convolution, we further introduce the other branch which utilizes Group Convolution (GConv) process the packed R, G1, G2, and B channels separately. Since the four groups may contribute differently to the demoiréing process, we further introduce learnable parameters modulate the four groups (i.e., channel-wise multiplication), shown in Fig. 5. Then, the color-mixed features and color-separated features are fused together via element-wise addition. In this way, we can combine the advantages of the two features.
Afterwards, we further introduce spatial modulation. The moiré patterns usually have a large spatial period. The receptive field of small convolution filter is limited. Therefore, propose to utilize the feature with large receptive field to modulate the feature with small receptive field. Inspired by [7]. we directly utilize the convolution based modulation to realize this process. As shown in Fig. 5, we first utilize the Linear operation (realized by x conv) to transform the fused features. Then, the top branch goes through depth-wis convolution (DConv) with large kernel (11 11), and the generated feature is used to modulate the feature in the bottom branch. Afterwards, the modulated feature further goes through Linear layer introduce cross-channel communication which absent for DConv. After channel and spatial modulation (CSMod), the color-separated feature and color-mixed feature are well fused and enhanced, which are then fed into the following Conv blocks for demoiréing reconstruction.
4.3 Loss Functions
We observe that two-stage training is better for our network. Therefore, we utilize two-stage training strategy. At the first stage, the baseline network (namely removing the color-separated feature and CSMod from Fig. 4) is trained with fl loss and perceptual loss. It is formulated as (for brevity,

---

R
G1
G2
B
Linear
Conv
Color-seperated Features
Linear
Leamable Paramelers
Color- mixed Features
Linear
Channel Modualtion
Spatial Modelation
Figure 5: Channel and spatial modulation (CSMod) block.
omit the subscript sRGB for Of and (
k=3
C.1 AullO* 11 + )ell)s(0'r)) - bhy - II
(I)
where 7% is the ground truth and represents the scale index. By represents the VGG features extracted the jth layer, and utilize the convl layer in implementation. X1 and d2 the weighting parameters. This stage can remove the moiré patterns to some extent but fail for hard samples. For example, if the moiré patterns be removed with only the color-mixed features. Therefore, in the second stage, we introduce the color-separated feature and CSMod to the baseline, and train the whole network. Since need to recover correct color tones in sRGB domain, similar to [34], we further introduce color loss. Namely, the second stage loss function is
strong for color channel, it is difficult to
Oui 10
L.2 Σ - - Xallvs((O*)) diy (T((
As(1 N
Σ
where k is the pixel index, and N is the pixel number. As, As. and As are the weighting parameters.
5
Experiments
5.1 Implementation Details and Datasets
For video-demoiréing, the weighting parameters A in Eq. and 2 are set to 0.5, 1, 5, 1, respectively. The first stage training starts with learning rate of 2e-4, which decreases to 5e-5 at the 37th epoch. The baseline reaches convergence after 40 epochs. In the second stage, the initial learning rate is 2e-4, which decreases to le-4 and 2.5e-5 at the 10th and 30th epochs. All the network parameters are optimized by Adam. The batch size to 14 and the patch size set to 256. Image demoriéing shares similar training strategies. The proposed method is implemented in PyTorch and trained with two NVIDIA 3090 GPUs.
For image demoiréing, we utilize the dataset constructed by [34], which contains raw image inputs. For video demoiréing, utilize the dataset constructed in Sec. which contains 300 video clips. We randomly select 50 video clips to serve as the testing set.
Table 1: Comparison with state-of thetthhhaarr art image demoiréing methods in terms average PSNR.
SSIM, LPIPS, and computing complexity. The best results are highlighted in bold.
Method
PSNR1
SSIM
LPIPS
Parameters (M)
GFLOPs
FHDee
23,630
0.8720
0.1540
13.59
353.37
RDNet
26.160
0.9210
0.0910
6.045
10.06
MBCNN
26.667
0,9272
0,0972
2.722
9.74
MBCNN
27.187
0,9332
0,0888
2,717
9.67
UHDM
26.513
0,9246
0,0911
5,934
17.56
U
27.078
0,9302
0.0812
5.925
17.40
Ours
27.261
0.9346
0.0748
5.327
21.65

---

Table 2: Comparison with state-of-the-art image and video demoiréing methods for video demorićing terms of average PSNR, SSIM, LPIPS, and computing complexity. The best result is highlighted in bold and the second best is underlined.
Method
PSNR
SSIM
LPIPS
Parameters (M)
GFLOPs
MBCNN
25,892
0.8939
0,1508
2.722
9.74
MBCNN
26.082
0.8984
0,1402
2.717
9.67
Image
UHDM
26.823
0,9078
0.1177
5,934
17.56
UHDM
25.642
0.8792
0.1232
5.925
17.40
WHrame-RDNel
26,970
0.9051
0,1176
6.640
34,31
MFrame-UTh UNet
26.185
0.8917
0.1368
32.286
39.01
VIMoiré
27.277
0.9071
0.1044
5,836
171.93
VDMoiré
27.747
0,9116
0.0995
5,838
43.02
Video
EDVR
26.785
0.8993
0.1172
2.857
990.74
EDVR
27.285
0.9064
0.1053
3.005
257.50
VRT
24.903
0.8728
0.1387
2.485
353.37
VRT
27.113
0.9034
0.1091
2.733
101.60
Ours
28.706
0.9201
0.0904
6.585
70.61
5.2 Comparison for Image Demoiréing
For image demoiréing, we compare with the raw domain demoiréing method [34] and the state-of-the
art sRGB domain image demoiréing methods, i.e., FHDe Net [6]. MBCNN [41], and UHDM [28]. All these methods are retrained on the raw image dmoiréing dataset [34]. For the most competitive methods MBCNN and UHDM, we further retrain them with raw inputs by revising (changing the input channel number and inserting upsampling layers) the network make adapt to raw inputs. The revised versions are denoted as MBCNN* and UHDM* Since PSNR is heavily affected by reconstructed color tones, SSIM and LPIPS [38] are good complementary measurements for comprehensive evaluation. As shown in Table 1, our method achieves the best results on all three measurements. Fig. presents the visual comparison results for the top six methods. can be observed that all the compared methods fail when deal with the regular and strong moiré patterns. In contrast, our method can remove them clearly.
5.3 Comparison for Video Demoiréing
To our knowledge, there is only one work, i.e., VDMoiré [3], dealing with video demoiréing. Therefore, we further include other video restoration methods and state-of-the-arr image demoiréing methods for comparison. Specifically, we introduce the benchmark CNN based and transformer based video super-resolution networks, i.c., EDVR [20] and VRT [8], for comparison. Since VRT very large, reduce its block and channel numbers for a fair comparison, and the revised version is denoted as VRT The original loss functions of EDVR and VRT are changed to our loss functions for better demoiréing performance. For image demoiréing, for comparison. Since all the compared methods are designed for sRGB inputs, we first retrain them Raw VDemoir dataset with sRGB domain pairs. Then, retrain them with raw-sRGB pairs. We utilize superscript to denote the raw input versions.
introduce MBCNN [41] and UHDM [28]
addition, we transform two image processing methods (RDNet [34] and UNet [16]) for video demoiréing incorporating the PCD module and fusion module. The revised versions are denoted as MFrame-RDDa and MFrame-UNet The inputs of the modules three consecutive frames.
Moiré (sRGB)
RONE
MBCNN
MBCNN
UHDM
UHDM
Ours
GT
Figure 6: Comparison with -fate-of-the-e image demoiréing methods.

---

Table 2 presents the quantitative comparison results. All the raw domain processing methods (except UHDM*) are better than their corresponding sRGB domain versions. All the image based demoiréing methods cannot achieve satisfied results for video demoiréing since they cannot utilize temporal correlations. For EDVR, since does not reconstruct multiscale results, it cannot remove large moiré patterns. VRT also cannot generate satisfied results since the shidt-window matching mechanism of VRT is not beneficial for large moiré removal. In addition, the revised multi-frame based raw image processing methods MFrame RDNet and MFrame UNet still cannot generate satisfied results. In contrast, our method generates the best demoiréing results, outperforming the second best method by nearly dB. The main reason is that our CSMod strategy can well utilize the complementary information of RGGB channels and can remove strong moiré patterns.
We also give the computing complexity comparison in terms of parameter numbers and GFLOPs in Table 2. can be observed that the image demoiréing methods consume small computing costs since they process the video frame-by-frame which also leads to worse demoiréing results. Our GFLOPs are smaller than the popular video restoration frameworks, such as EDVR and VRT, which indicates our method is efficient video demoiréing method.
Fig. presents the visual comparison results for one video frame. be observed that our method can remove the moiré patterns clearly while the compared methods all fail for this hard sample. More results are provided in our appendix.
Moid (sRGB)
MIKONN
UHIM
VUMoie?
EDVR*
VRT
Unrs
Figure 7: Comparison for video demoiréing.
5.4 Ablation Study
Table 3: Ablation study results by adding the proposed modules one by one. The best re-
Table 4: Ablations on loss functions and the multi-scale constraint. The best results are highlighted in bold.
sults are highlighted in bold.
Variant
PSNR
SSIM
LPIPS
Variant
PSNR1
SSIM
LPIPS
Baseline
27.963
0.9116
0.0914
w/o VGG Loss
28.069
0.9166
0.1160
Baseline+SMMod
28.439
0.9175
0.0939
w/o Color Loss
28.642
0.9169
0.0931
Baseline+SMMdd
28.498
0.9182
0.0957
w/o Multi scale Loss
27.455
0,9124
0.1041
Baseline+CSMod
28.706
0.9201
0.0904
Complete
28.706
0.9201
0.0904
Ablation on SMod and CSMod. We perform ablation study by adding the proposed modules one by one. The first variant is our baseline network. The second is introducing the proposed spatial modulation into the baseline network. Namely we only keep the color-mixed features in the channel modulation module in Fig. 5. The third is introducing SMod and replacing the color-separated feature in Fig. with another color-mixed feature (denoted SMod The fourth is our complete version, namely Buseline+CSMod. As shown in Table 3, the SMod improves the demoiréing performance by 0.48 dB. One reason that the SMod can take advantage of large receptive field and the other reason is that we utilize the second stage training loss to update SMod, which is beneficial for color recovery. Compared with SMod, S only achieves little improvement, which demonstrates that the color-separated features are essential. Compared with SMod, the proposed CSMod can further improve 0.26 dB.
Ablation on Loss Functions. We further conducted ablation study regarding perceptual loss (VGG Loss), color loss (Color Loss), and multi-scale constraints (Multi-scale Loss), and the results are summarized in Table 4. When the corresponding loss is removed, there is a noticeable decrease in all

---

evaluation metrics. Perceptual loss places constraints on deep-level features of images, which helps recover structural information, such shapes and textures. Color loss, on the other hand, focuses more on the global color of images. Recaptured screen images often suffer from weakened attributes like brightness, contrast, color saturation, and details. 'The constraint provided by I.1 loss alone might not be sufficient. Incorporating perceptual loss and color loss can contribute to better image reconstruction quality. Given the multi-scale characteristies of moiré patterns, imposing multi-scale constraints is beneficial for moiré removal and image reconstruction. Note that, the multi-scale constraints are widely utilized benchmark demoiréing models, such as MBCNN, UHDM, and VDMoiré. In addition, the perceptual loss is also included in all of our compared methods (except MBCNN, which utilizes a more advanced loss function).
5.5
Generalization
The image demoiréing experiment conducted on the dataset provided by [34], whose test set contains two device combinations that not included in the training set. Therefore, these test images (about 1/4 of the whole test set) to evaluate the generalization ability of different models. As shown in Table our method still achieves the best results in terms of SSIM and LPIPS, and the PSNR value is only slightly inferior to U Note that, for our method, the test camera-screen combination exists in the training set, the color tone of the test images can be well recovered. Otherwise, the recovered image may have color cast. The main reason is that our network also performs the ISP process, and the ISP process indeed different for different cameras. Fortunately, the moire patterns can still be removed. Therefore, our LPIPS value is still small. In summary, our method has the best generalization ability terms of moiré removal.
utilize
Table Comparison of the generalization ability of different models by evaluating with the screen- camera combinations that are not included in the training set. The best results are highlighted in bold and the dataset the image demorićing dataset [34].
Method
PSNR
SSIM
LPIPS|
FHDe
22.840
0.8968
0.1530
RDNet
24.798
0.9280
0.0846
MBCNN
24.726
0.9253
0.1017
MBCNN
24.659
0.9314
0.0925
UHDM
24.679
0.9239
0.0955
U
25.769
0.9310
0.0759
Ours
25.719
0.9333
0.0727
Conclusion
In this paper, we propose a novel image and video demoiréing network tailored for raw inputs. We propose efficient channel modulation block to take advantages of repor-separated features and color-mixed features. The spatial modulation module helps our model utilize large receptive field information with fewer computation costs. In addition, construct Raw VDemoiré dataset to facilitate research this area. Experimental results demonstrate that our method achieves the best results for both image and video demoiréing.
7 Broader Impact
Our work is the first exploring raw video demoiréing and we construct the first Raw VDemoiré dataset. To solve the temporal confusion problem during video recapturing, propose efficient temporal alignment method, which can be extended to other screen recapturing scenarios. Previous works usually omit the differences between and sRGB inputs. Our work an exploratory work that takes advantage of color-separated features in raw domain and we expect more works along this direction. However, we would like to point out that the demoiréing results may be used for face-spoofing since the moiré detection method cannot identify our result. But on the other hand, our method can imitate the attacks for face recognition, which can help train robust antispoofing methods.
10

---

References
[1] Chen Chen, Qifeng Chen, Minh N Do, and Vladlen Kolunn Seeing motion in the dark. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3185-3194, 2019.
[2] Chen Chen, Qifeng Chen, Jia Xu, and Vladlen Koltun. Learning to see in the dark. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3291-3300, 2018.
[3] Peng Dai, Xin Yu, Lan Ma, Baoheng Zhang, Jia Li, Wenbo Li, Jiajun Shen, and Xiaojuan Qi. Video demoireing with relation-based temporal consistency. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17622-17631, 2022.
[4] Hansen Feng. Lizhi Wang. Yuzhi Wang, Hanqiang Fan, and Hua Huang. Learnaillity enhancement for low-light raw image denoising: A data perspective. JEEF Transactions on Pattern Analysis and Machine Intelligence, 2023.
[5] Bin He, Ce Wang, Boxin Shi, and Ling-Yo Yu Duan. Mop moiré patterns using mopnet. In Proceedings of the IEEE International Conference on Computer Vision, pages 2424-2442 2019.
[6] Bin He, Ce Wang. Boxin Shi, and Ling-Yu Yu Duan. FHDe2Net full high definition demoiréing network. In Proceedings of the European Conference Computer Vision, pages 713-729. Springer, 2020.
[7] Qibin Hou, Cheng- Ze Lu, Ming-Ming Cheng, and Jiashi Feng. Conv2former: A simple transformer-styy style convnet for visual recognition. arXiv preprint arXiv:2211. 1943, 2022.
[8] Jingyun Liang. Jiezhang Cao, Yuchen Fan. Kai Zhang, Rakesh Ranjan, Yawei Li, Radu Timofte, and Luc Van Gool. Vrt: A video restoration transformer. arXiv preprint arXiv:2201.122
2022.
[9] Fanglei Liu, Jingyu Yang. and Huanjing Yue. Moiré pattern removal from texture images via low-rank and sparse matrix decomposition. In 2015 Visual Communications and Image Processing (VCIP), pages 1-4.
IEFF, 2015.
[10] Jiaming Liu, Chi-Hao Wu, Yuzhi Wang. Qin Xu, Yuqian Zhou, Haibin Huang. Chuan Wang, Shaofan Cai. Yifan Ding, Haoqiang Fan, et al. Learning raw image denoising with bayer pattern unification and bayer preserving augmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2070-2077, 2019.
[11] Lin Liu, Jianzhuang Liu, Shanxin Yuan, Gregory Slabaugh, Ales Leonardis, Wengang Zhou, and Qi Tian. Wavelet-based dual-branch network for image demoiréing. In Proceedings of the European Conference on Computer Vision, pages 86- 102. Springer. 2020.
[12] Lin Liu, Shanxin Yuan, Jianzhuang Liu, Liping Bao, Gregory Slabaugh, and Qi Tian. Self-adaptively learning to demoiré from facused and defocused image pairs. Advances in Neural Information Processing
Systems. 33:22282-22292 2020.
[13] Xiaohong Liu, Kangdi Shi, Zhe Wang, and Jun Chen. Exploit camera raw data for video super-resolution via hidden markov model inference. JEEE Transactions on Image Processing, 30:2127-21444 2021. [14] Yuzhen Niu, Zhihua Lin, Wenxi Liu, and Wenzhong Guo. Progressive moire removal and texture complementation for image demoireing. IEEE Transactions on Circuits and Systems for Video Technology. 2023.
[15] Federico Perazzi, Jordi Pont-Tuset, Brian McWilliams, Luc Van Gool, Markus Gross, and Alexander Gorkine-Homung. benchmark dataset and evaluation methodology for video object segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 724-732, 2016.
[16] Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assistee Intervention MICCAI 2015:
18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pages 234-241.. Springer, 2015.
[17] H. Siddiqui. M. Boutin. and C. Bouman. Hardware-friendly descreening. IEEE Transactions on Image Processing (TIP). 19(3):746-757, 2010.
[18] Bin Sun, Shutao Li, and Jun Sun. Scanned image descreening with image redundancy and adaptive filtering.
IEEE Transactions on Image Processing (TIP), 23(8):369 -7100, 2014.
[19] Yujing Sun, Yizhou Yu, and Wenping Wang. Moiré photo restoration using multiresolution convolutional
neural networks. IEEE Transactions on Image Processing. 27(8):160-4417
2018.
[20] Xintao Wang. Kelvin CK Chan, Kc Yu, Chao Dong, and Chen Change Loy. Edvr: Vidco restoration with
enhanced deformable convolutional networks. In Proceedings of the IEEF/CVF Conference on Computer Vision and Patiern Recognition Workshops, pages 0-0, 2019.
[21] Yuzhi Wang. Haibin Huang, Qin Xu, Jiaming Liu, Yiqun Liu, and Jue Wang. Practical deep raw image denoising on mobile devices. Proceedings of the European Conference on Computer Vision, pages 1 1-16, 2020.
[22] Zhendong Wang, Xiaodong Cun, Jianmin Bao, Wengang Zhou, Jianzhuang Liu, and Houqiang L.i. Uformer: A general u-shaped transformer for image restoration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 17683-17693, 2022.
[23] Kaixuan Wei, Ying Fu, Jiaolong Yang, and Hua Huang. physics-based noise formation model for extreme low-tight raw denoising. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2758-2767, 2020.
[24] Philippe Weinzaepfel. Jerome Revaud, Zaid Harchaoui, and Cordelia Schmid. Deepttow: Large displace- ment optical flow with deep matching. In Proceedings of the IEEE International Conference on Computer Vision, pages 1385-1392, 2013.
11

---

[25] Fuzhi Yang. Huan Yang, Jianlong Fu, Hongtao Lu, and Baining Guo. Learning texture transformer network for image super-resolution. In Proceedings of the IEEE/CVF conference on computer vision and pattern recogmition, pages 5791-5800, 2020.
[26] Jingyu Yang. Fanglei Liu, Huanjing Yue, Xiaomei Fu, Chunping Hou, and Feng Wu. Textured image demoiréing via signal decomposition and guided filtering. IEEE Transactions on Image Processing.
26(7))352--3541, 2017.
[27] Jingyu Yang, Xue Zhang. Changrui Cai, and Kun Li. Demoinéing for screen-shot images with multi-channel layer decomposition. In 2017 IEEE Visual Communications and Image Processing (VCIP), pages 1-4, 2017.
[28] Xin Yu, Peng Dai. Wenbo Li, Lan Ma, Jiajun Shen, Jia Li, and Xiaojuan Qi. Towards efficient and scale-robust ultra-tigh-deffiition image demoiréing. In Computer Vision-ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23 27, 2022, Proceedings, Part XVIII. pages 646 662. Springer, 2022.
[29] Shanxin Yuan, Radu Timofte, Ales Leonardis, and Gregory Slahaugh. NTIRE 2020 challenge on image demoiréing: Methods and results. In Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition, pages 460-461. 100-611,2222 2020.
[30] Shanxin Yuan, Radu Timofte, Gregory Slabaugh, Aleš Leonardis, Bolun Zheng, Xin Ye, Xiang Tian, Yaowu Chen, Xi Cheng. Zhenyong Fu, et al. AIM 2019 challenge image demoiréing: Methods and results. In Proceedings of the IEEE International Conference
Computer Vision, pages 3534-3545, 2019. [31] Huanjing Yue. Cong Cao, Lei Liao, Ronghe Chu, and Jingyu Yang. Supervised raw video denoising with benchmark dataset on dynamic scenes. Proceedings of the IEEE Conference on Computer Vision and
Pattern Recognition, pages 2301-2310, 2020.
[32] Huanjing Yue, Cong Cao, Lei Liao, and Jingyu Yang. Rvideformer: Efficient raw video denoising transformer with a larger benchmark dataset. arXiv preprint arXiv:2305.0076 2023.
[33] Huanjing Yue, Yijia Cheng, Fanglong Liu, and Jingyu Yang. Unsupervised moiré pattern removal for recaptured screen images. Neurocomputting 456:352- 2021.
[34] Huanjing Yue, Yijia Cheng, Yan Mao, Cong Cao,, and Jingyu Yang. Recaptured screen image demoirêing
in raw domain. JEEE Transactions on Multimedia, 2022.
[35] Huanjing Yue, Yan Mao, Lipu Liang, Hongteng Xu, Chumping Hou, and Jingyu Yang. Recaptured screen image demoiréing. IEEE Transactions
Circuits and Systems for Video Technology, 31(1):49 60. 2021. [36] Huanjing Yue, Yubo Peng. Biting Yu, Xuanwu Yin, Zhenyu Zhou, and Jingyu Yang. Hdr video re- construction with large dynamic dataset in raw and srgb domains. arXiv preprint 344447330,
2023.
[37] Huanjing Yue, Zhiming Zhang, and Jingyu Yang. Real-rawvsr: Real-world raw video super-resolution with benchmark dataset. In Computer Vision-ECCC 2022: 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part VI, pages 608-624. Springer, 2022.
[38] Richard Zhang. Phillip Isola, Alexci A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE Conference on Computer Vision and Pottern Recognition. pages 996,595,55 2018
[39] Xuaner Zhang, Qifeng Chen, Ren Ng, and Vladien Koltun. 7oom to learn, learn to zoom. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pages 3762-3770. 2019. [40] Yuxin Zhang, Minghao Lin, Xunchao Li, Han Liu, Guozhi Wang, Fei Chao, Shuai Ren, Yafei Wen, Xiaoxin Chen, and Rongrong Ji. Real-time image demoireing on mobile devices. ICLR, 2023.
[41] Bolun Zheng, Shanxin Yuan, Gregory Slabaugh, and Ales Leonardis. Image demoiréing with learnable bandpass filters. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3636-3606 3066-665,220 2020.
[42] Bolun Zheng, Shanxin Yuan, Chenggang Yan, Xiang Tian, Jiyong Zhang, Yanqi Sun, Lin Liu, Alek Leonardis, and Gregory Slabaugh. Learning frequency domain priors for image demoireing. IEEE Transactions on Puttern Analysis and Machine Intelligence,
(111))777-7711 2021.
[43] Yunhao Zou, Chenggang Yan, and Ying Fu. Rawhdr: High dynamic range image reconstruction from a single raw image. In Proceedings of the IEEE/CVF International Conference Computer Vision, pages 12334- 12344, 2023.
12


Reviews:
[{"review_id": "0B1HTI4gVx", "reviewer": "Reviewer_AQcs", "summary": "This paper proposed an image and video demoiréing network tailored for raw inputs.  The authors constructed the first real raw video demoiréing dataset with temporal and spatial aligned pairs and designed a network tailored with two branches including one branch to share feature enhancement and the other branch to modulate different color groups. Experimental results demonstrate that the proposed method achieves the best results on both raw video and image demoriéing datasets.", "questions": "-Are the image observations of different raw image channels in Figure 1 representative?", "limitations": "Yes, the authors have discussed the limitations and potential negative societal impact of their work.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "+This paper first explores raw video demoiréing and constructs the first RawVDemoiré dataset.\n+In this paper, several strategies are designed to solve the problem of time synchronization and spatial alignment between the original image and the shot image in data acquisition.\n+The experimental results presented in this paper demonstrate the effectiveness of the collected dataset to improve the performance of the demoiréing network.", "weaknesses": "-Although the experimental results demonstrate the effectiveness of the collected dataset, compared with UHDM*, the performance of the proposed network has no significant advantages. Neither numerical comparison nor qualitative results can clearly show the advantages of the network.\n-This paper does not analyze the performance limits of the proposed network, that is, the performance and advantages in the face of extremely severe moire patterns.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your nice summary and positive comments on our datasets. In the following, we give detailed responses to your comments.\n> -Although the experimental results demonstrate the effectiveness of the collected dataset, compared with UHDM*, the performance of the proposed network has no significant advantages. Neither numerical comparison nor qualitative results can clearly show the advantages of the network.\n\nWe would like to point out that our method does outperform UHDM* in both qualitative and quantitative measurements. Due to the page limits, our main paper only provides two visual comparisons. In Fig. 6, although the moiré removal performance of our method is similar to that of UHDM*, our method works better in color reproduction. In Fig. 7, our method can remove the moiré patterns clearly but there are still obvious moiré patterns in the result of UHDM*. In addition, the visual comparison results in Fig. 3 and Fig. 4 of the supplementary file also demonstrate that the results of UHDM* have color cast and the reconstructed edges are kind of blurry. In contrast, our method can remove the moiré patterns clearly and recover sharp edges and correct colors.  For comprehensive comparison, we further present two examples in Fig. R1.\n\nFor quantitative comparison (Table 1 and Table 2 in the main file, which is also summarized in the following Table R1), our method outperforms UHDM* for both image and video demoiréing. Since it is difficult to use one measurement to evaluate the demoiréing and detail reconstruction performance, we use PSNR, SSIM, and LPIPS for comparison. Among these metrics, LPIPS is more effective at reflecting the quality of moiré pattern removal. For image demoiréing, our method outperforms UHDM* by 0.0064 in terms of LPIPS, which is considered a substantial improvement in relevant visual tasks. For video demoiréing, our method outperforms UHDM* with a large margin. One reason is that we utilized multiple frame correlations. The other important reason is that the moiré patterns and color distortion in the video demoiréing dataset are more heavier than that in image demoiréing dataset, and our method is much better in dealing with these distortions than UHDM*.  \n\nTable R1: Comparison between our method and UHDM*. \n| Category | Methods | PSNR | SSIM | LPIPS |\n|  ----  | ----  |  ----  | ----  |  ----  |\n| Image Demoiréing | UHDM* | 27.078 | 0.9302 | 0.0812 |\n| Image Demoiréing | Ours | **27.261** | **0.9346** | **0.0748** |\n| Video Demoiréing | UHDM* | 25.642 | 0.8792 | 0.1232 |\n| Video Demoiréing | Ours | **28.706** | **0.9201** | **0.0904** |\n\n> -This paper does not analyze the performance limits of the proposed network, that is, the performance and advantages in the face of extremely severe moiré patterns.\n\nThanks for your suggestion. When the image has high color saturation and the moiré pattern contrast is strong, our method may cannot remove the patterns clearly. Fig. 1 in the main paper belongs to this case. Our method can remove the moiré patterns to some extent, while some compared methods (such as RDNet and MBCNN) failed in dealing with this hard example. Fig. R2 also gives an example. When dealing with this kind of challenging example, our method still works the best. In the final version, we will introduce the discussion and give more examples about this limitation. \n\n> -Are the image observations of different raw image channels in Figure 1 representative?\n\nYes, our observation that different channels have different moiré patterns is representative. In Fig. R3, we provide eight examples of the moiré frames and give their DCT spectrum. The moiré patterns usually appear as nearly periodic stripes, which implies that the moiré patterns will be represented as strong peak spots in the DCT spectrum. It can be observed that the DCT spectrum of different raw channels in the same image are different.  This is consistent with our observations in the image domain that different raw channels have different moiré patterns. \n"}, {"review_id": "JoZ3R8Eb9Z", "reviewer": "Reviewer_pajB", "summary": "This paper addresses the problem of image/video demoireing through raw image data. The paper shows some observations, such as, the moire pattern in different color channels of the raw inputs have different properties, and pursues how to used this properties for raw image/video demoireing. It proposes to use channel and spatial modulations. To deal with this problem, the paper proposes a new dataset. The proposed algorithm achieves state-of-the-art performance.\n", "questions": "none", "limitations": "see weaknesses", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The paper has some nice observations, such as the moire patterns in different color channels of the raw inputs have different properties. It tries to exploit this property and come up with a new algorithm using channel and spatial modulations. The algorithm achieves state-of-the-art performance.", "weaknesses": "Although the paper shows some nice observations, there is no in-depth analysis why these observations happens, e.g., why the R, G1, G2, and B channels have different moire patterns. Based on Figure 1, the moire shapes looks quite similar to me, only the intensities are different. My guess is that the grid for R, G1, G2, B are the same, only with a pixel shift in locations, and the color filters will filter out the light for different colors, and thus have different intensities, but I am not exactly sure. The paper should provide more insight into this aspect.\n\nThe paper put a lot of efforts into building a new video raw moire dataset, and it is actually the best selling point of this paper from my point of view. Why is this necessary?  If one want to deal with raw image/video demoireing, is not it easier to directly convert the sRGB images to RAW images, and spend moire efforts on designing better architectures?\n\nI am not quite convinced by the novelty in the network design, it is simply follow the pipeline of feature extraction, alignment, temporal fusion and reconstruction. It combines with a few modules, such as PCD alignment, but I am not convinced if this is novel.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your nice summary and positive comments on our novelty and contributions. In the following, we give detailed responses to your comments.\n\n> Although the paper shows some nice observations, there is no in-depth analysis why these observations happens, e.g., why the R, G1, G2, and B channels have different moiré patterns. Based on Figure 1, the moiré shapes looks quite similar to me, only the intensities are different. My guess is that the grid for R, G1, G2, B are the same, only with a pixel shift in locations, and the color filters will filter out the light for different colors, and thus have different intensities, but I am not exactly sure. The paper should provide more insight into this aspect.\n\nThanks for your careful review. We would like to point out that the moiré shapes in R, G1, G2, and B channels are different. In Fig. R4, we present the zoom-in version of the moiré patterns in different raw channels for better observation. For the exampled image, its R channel only contains vertical and horizontal stripes with the same regular scale. In contrast, its G2 channel contains fine-scale moiré patterns besides the regular scale ones. To make this clearer, we further present the DCT spectrum of the four channels. The moiré patterns usually appear as nearly periodic stripes, which implies that the moiré patterns will be represented as strong peak spots in the DCT spectrum. As shown in the second row of Fig. R4, the R and G1 channels only have peak spots at the left-up corner, which represent coarse-scale moiré patterns and this is consistent with the observation in the image domain. Meanwhile, the G2 and B channels have peak spots at left-up, right-up, and left-down corners. It means that the two channels have both coarse-scale and fine-scale moiré patterns, which is consistent with the observation in the image domain. Note that, all the images are normalized (the DCT spectrum is also calculated from normalized image) to avoid the effects of different intensities of different CFA channels.  \n\nThe next important question is why they have different moiré patterns. We would like to point out that the moiré patterns are affected by the screen, camera, viewpoint, and image content. As you mentioned, there is a pixel shift in locations across the Bayer pattern channels in the raw domain. In other words, the viewpoints of the R, G1, G2, and B channels are different. During data acquisition, even one-pixel shift can have a significant impact on the period and shape of moiré patterns, rather than being just a one-pixel displacement. On the other hand, with the same camera, screen, and viewpoints but different contents, the generated moiré patterns are also different. For the R, G1, G2, and B channels of the same image, their contents are also different to some extent (due to different spectrum responses of the color filter). Therefore, the moiré patterns of the four channels are different. In our final version, we will add these deep analysis for our observations.  \n\n> ...Why not directly convert the sRGB images to RAW images, and spend moiré efforts on designing better architectures?\n \nWe would like to point out that it is difficult to inverse a clean RGB image into raw domain accurately since the ISP process is nonlinear and the raw image has wider bit depth than that of the sRGB image. If the RGB image contains moiré patterns, it will be more difficult to reproduce the moiré patterns captured in the original raw domain.  In Fig. R4, we present the DCT spectrum of the moiré images across different channels in both raw and sRGB domain. As mentioned in the response to your first question, the peak spots in the DCT spectrum represent the moiré patterns. The four channels in the raw domain have different peak spots, which means the moiré distributions are also different. Meanwhile, the DCT spectrum of the R, G, and B channels of the RGB image are similar. The reason is that the complex ISP process mixes these moiré patterns. Then we perform inverseISP (using the implementation provided by CycleISP) on the moiré RGB image, and the DCT spectrum of the inversed raw image are presented on the fourth row. It can be observed that they still have similar peak spots. In other words, the distinct moiré patterns in different raw channels cannot be recovered. Therefore, utilizing the real captured raw images for demoiréing network training is better than utilizing inversed (synthesized) raw images. In addition, even for denoising, utilizing real captured raw is also better than using synthesized raw, as shown in [29].  Utilizing inverseISP for dataset generation is a good topic for further exploration but our work did not focus on this point.  Our constructed dataset not only benefits the network training but also provides a real test set to evaluate different demoiréing methods. \n\n>I am not quite convinced by the novelty in the network design, it is simply follow the pipeline of feature extraction, alignment, temporal fusion and reconstruction. It combines with a few modules, such as PCD alignment, but I am not convinced if this is novel.\n\nThe pipeline of feature extraction, alignment, temporal fusion and reconstruction is our baseline other than our contribution. The main novelty of this work is the design of channel modulation and spatial modulation, which are devised based on our observations that moiré images in the raw domain channels have different properties. If we utilize traditional convolution to recover shared structures, the moiré patterns in different channels are also spread. Therefore, we propose to use the complementariness between color-separated features and color-mixed features. To our knowledge, in the literature, there is no work giving this observation and no work exploring the complementariness of color-separated features and color-mixed features.  We believe this is a valuable contribution, as it poses new opportunities to solve the raw image (video) demoiréing problem.\n"}, {"review_id": "SFMrA9vTHM", "reviewer": "Reviewer_irQe", "summary": "This paper proposes an image and video demoiréing network tailored for raw inputs. It introduces a color-separated feature branch, and it is fused with the traditional feature-mixed branch via channel and spatial modulations. It builds the first well-aligned raw video demoiréing (RawVDemoiré) dataset and achieves state-of-the-art performance for both image and video demoriéing.", "questions": "see Weaknesses. ", "limitations": "No", "rating": 4, "confidence": 5, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "A raw demoireing dataset is proposed.\nExperimental results demonstrate that the method achieves the best results on both raw video and image demoriéing datasets.", "weaknesses": "First of all, the network structure is not very novel. The modulation of channels is similar to channel attention mechanisms, and cross-channel and group modulation are also common practices. Secondly, the need for a large receptive field has been discussed in other articles on moiré pattern removal. Multiscale and large convolution kernels, as well as dilated convolutions, are also common approaches. From the experimental results in Table 3, it can be observed that the baseline (27.96) outperforms VDMoiré* (27.74) and vrt* (27.11). Therefore, I believe it is worth further exploring the effectiveness of the approach in details.\n\n'Recaptured screen image demoiréing in raw domain' demoires in raw domain. I think it would be beneficial to transform it into a multi-frame version (e.g., by adding a PCD module) and then conduct a comparison. The paper can also compare some end-to-end methods that specifically handle raw domain images.\n\nI am not clear about the significance of the experiments in Table 4. I believe that comparing the results of a single channel alone may not be as informative as the complete results. Therefore, I suggest moving this experiments from Table 4 to supplementary materials. Instead, it would be more helpful to include experiments that investigate the impact of loss functions and multi-scale constraints.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your nice summary and positive comments on our datasets and SOTA performance. In the following, we give detailed responses to your comments.\n\n> First of all, the network structure is not very novel. The modulation of channels is similar to channel attention mechanisms, and cross-channel and group modulation are also common practices. Secondly, the need for a large receptive field has been discussed in other articles on moiré pattern removal. Multiscale and large convolution kernels, as well as dilated convolutions, are also common approaches.  ...\n\nOur \"channel modulation\" operation is different from the traditional \"channel attention\" operation. In Table R2, we performed an ablation experiment by replacing our channel modulation with traditional channel attention. Despite channel attention utilizing more parameters (fully connected layers) to learn channel weights, its performance is inferior to that of our method. Fig. R5 presents the learned weights of channel modulation and channel attention. It can be observed that the channel attention weights are similar, which means there are no large differences. The main reason is that it needs a sigmoid layer to generate these coefficients. Different from it, our modulation coefficients are learned by setting them to be learnable parameters. The learned modulation coefficients have a large variance, which implies that different channels contribute very differently in the following.  Note that, our main novelty is jointly utilizing the color-separated and color-mixed features based on our observation that different channels have different moiré patterns in raw domain. **To our knowledge, there is no work giving this observation and no work exploring the complementariness of color-separated features and color-mixed features for image/video demoiréing. We believe this is a valuable contribution, as it poses new opportunities to solve the raw image (video) demoiréing problem.**\n\nAs you mentioned, utilizing large receptive field for image demoiréing, such as multi-scale, large kernels, and dilated convolution, is a common strategy. In this work, we adopted a light-weight transformer-style convnet [6] to realize the spatial modulation task. This operation has only been utilized in visual recognition and our work is the first that transforms it to low-level vision tasks.  In this way, our spatial modulation not only increases the receptive field but also reduces the parameter and computation costs. \n\nAs shown in Table 3 in the main paper, our CSMod strategy yields substantial gains on top of the baseline (e.g., nearly 0.8 dB increase in PSNR). We believe our work can inspire more works exploring the complementariness between color-separated and color-mixed features. \n\nTable R2: Comparison between our channel modulation and traditional channel attention. \n| Methods | PSNR | SSIM | LPIPS |\n| ---- | ---- | ---- | ---- |\n| channel attention | 28.492 | 0.9170 | 0.0987 |\n| channel modulation | **28.706** | **0.9201** | **0.0904** |\n\n> 'Recaptured screen image demoiréing in raw domain' demoirés in raw domain. I think it would be beneficial to transform it into a multi-frame version (e.g., by adding a PCD module) and then conduct a comparison.  The paper can also compare some end-to-end methods that specifically handle raw domain images.\n\nThanks for your suggestion. We have incorporated the PCD module and fusion module into RDNet, transforming it into a multi-frame version. The results are presented in Table R3. Our method consistently outperforms RDNet on all the three metrics. Despite RDNet being a raw domain approach, it did not consider the moiré differences in raw domain channels and it merely adjusted the channel number of the first convolution layer to fit the raw inputs, without any other tailored operations for raw data. In contrast, our method is tailored to the distinctive channel distribution properties of raw domain moiré patterns, and our modulation strategy leverages the inherent advantages of raw domain data more effectively.\n\nSince there is no other raw domain demoiréing methods, we further utilized UNet, a raw domain method that performed well in the experiments of RDNet, and adapted it to a multi-frame version. The results of these experiments are presented in Table R3, and our method also outperforms the multi-frame UNet approach. \n\nTable R3: Comparison between multi-frame based raw video demoiréing. \n| Methods | PSNR | SSIM | LPIPS |\n| ---- | ---- | ---- | ---- |\n| Multi-frame RDNet | 26.970 | 0.9051 | 0.1176 |\n| Multi-frame UNet | 26.185 | 0.8917 | 0.1368 |\n| Ours | **28.706** | **0.9201** | **0.0904** |\n\n> About experiments that investigate the impact of loss functions and multi-scale constraints.\n\nThanks for your suggestions. We will move Table 4 to the supplementary file and incorporate ablation experiments about loss functions and multi-scale constraints, as shown in Table R4. When the corresponding loss constraints were removed, there was a noticeable decrease in all evaluation metrics, which demonstrates the effectiveness of the hybrid loss functions and multi-scale constraints. Note that, the multi-scale constraints are widely utilized in benchmark demoiréing models, such as MBCNN [36], UHDM [26], and VDMoiré [3]. In addition, the perceptual loss is also included in all of our compared methods (except MBCNN, which utilizes a more advanced loss function). In summary, our method outperforms benchmark methods mainly due to our fusion of color-separated and color-mixed features and the proposed CSMod, which are our key contributions. \n\nTable R4: Ablation study about our loss functions and the multi-scale constraints.\n|Methods|PSNR|SSIM|LPIPS|\n|----|----|----|----|\n| w/o perceptual loss | 28.069 | 0.9166 | 0.1160 |\n| w/o color loss | 28.642 | 0.9169 | 0.0931 |\n| w/o multi-scale constraints | 27.455 | 0.9124 | 0.1041 |\n| Complete | 28.706 | 0.9201 | 0.0904 |\n"}, {"review_id": "t1N0OHHaeA", "reviewer": "Reviewer_piUN", "summary": "This paper aims at tackle the problem of recaptured raw screen image/video demoiréing. A deep demoireing network with a two-branch structure is proposed, where one branch is a cross-channel convolution for shared feature enhancement and the other is a color group convolution with learnable parameter modulation for different color groups. By fusing the two branch features together, disorder patterns are removed, and the shared visual structure is restored. Furthermore, the paper constructs the first real-world raw video demoireing dataset with temporal and spatial alignment pairs for efficient temporal alignment by inserting alternate patterns. Extensive experiments are conducted to demonstrate the performance of the proposed approach.", "questions": "Could the authors provide the results and comparison on the SRGB domain?", "limitations": "As discussed in the paper, like existing methods, the temporal consistency is not introduced in the proposed one.", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "1. The first raw video demoireing dataset is constructed, which benefits the research of related studies. \n2. It provides interesting observations: 1) moiré pattern characteristics are analyzed in the raw domain; 2) moiré pattern intensity is different in different color channels.\n3. The data collection technique used in this paper solves the problem of frame misalignment in video collection.\n4. The double-branch structure of the proposed network can extract both the color-mixed features and the color-separated features, which capture the moiré features more fully.\n5. The experiments are solid and comprehensive.\n6. The paper is well written.\n", "weaknesses": "1. The proposed network model does not show an advantage in model size.\n2. For image demoireing, the performance gain is relatively small compared to MBCNN*, but the number of references is twice as large.\n3. The two-branch structure has no rich interactions, but the network needs two stages for training.\n4. The data set produced in this paper includes both raw domain and srgb domain, but only raw domain is used in the experiment, while most of the compared methods are designed based on the SRGB domain.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your nice summary and positive comments on our contributions and datasets. In the following, we give detailed responses to your comments.\n\n> The proposed network model does not show an advantage in model size.\n\nWe agree with you that our method does not have advantage in model size. In real cases, the inference speed and computing costs are more important metrics since a small model size (the model sizes in our comparisons are all smaller than 14M) does not influence the inference speed and power dissipation. As shown in Table 2, our method demonstrates a lower GFLOPs compared to other video demoiréing methods. \n\n> For image demoiréing, the performance gain is relatively small compared to MBCNN*, but the number of references is twice as large.\n\nThanks. We agree with you that our model is larger than MBCNN* and PSNR gain over MBCNN* is small. However, for image demoiréing, the LPIPS metric is more sensitive in measuring the quality of moiré removal. As shown in Table 1, our LPIPS is 0.014 lower than that of MBCNN*, which is a substancial improvement in similar visual tasks. Furthermore, as evident from Figure 6 and the visual results provided in Fig. 3 and Fig. 4 of the supplementary file, the results of MBCNN* still contain noticeable moiré residuals. In contrast, our method can remove these moiré patterns clearly. We will give more results in our website to demonstrate the visual improvements of our method over compared methods. \n\n> The two-branch structure has no rich interactions, but the network needs two stages for training.\n\nThanks! In this work, we observe that the moiré patterns in different raw channels have different properties. Therefore, we extract color-mixed features and color-separated features, and propose channel modulation to fuse the two features together. Our solution is only a simple strategy to utilize the two kinds of features. A rich interaction may bring more gains, and we leave this as our future work. We find that two-stage training is better than one-stage training since our modulation operation requires a relatively comprehensive baseline feature representation. Fortunately, this does not influence the inference stage, which is more important in real applications. We will add these discussions in our final version. \n\n> The data set produced in this paper includes both raw domain and srgb domain, but only raw domain is used in the experiment, while most of the compared methods are designed based on the SRGB domain.\n\nWe produced sRGB domain pairs due to two reasons.  First, most compared moiré removal methods are proposed with sRGB domain inputs. Our sRGB domain pairs can facilitate the comparison with these methods. Besides, we selected well-performing sRGB-domain demoiréing methods and retrained them with raw inputs. Although the performance is improved with raw inputs, they are still inferior to our methods. Second, our sRGB domain pairs can facilitate subsequent research related to sRGB-domain demoiréing and the raw-RGB moiré pairs can also facilitate inverseISP tasks. For our method, we only utilized raw inputs since our method is built upon our key observation that the moiré patterns in different raw channels are different. \n\n> Could the authors provide the results and comparison on the SRGB domain?\n\nOur channel modulation strategy was designed based on the observation that the moiré patterns in different raw channels are different.  However, moiré patterns in the sRGB domain do not exhibit such characteristics (refer to the comparison between the second and third rows in Figure R4). Our method is tailored to raw domain data and is not suitable with sRGB domain inputs. Therefore, we did not provide the results on the sRGB domain. \n"}, {"review_id": "r8NRkgPc2s", "reviewer": "Reviewer_tgX2", "summary": "The paper proposed a neural algorithm to remove the moire pattern in an image/video when capturing a display. Since it is observed moire patterns are simpler in raw signals, the proposed method starts from raw data and applies channel and spatial modulations targeting moire pattern properties in the raw domain. A new dataset with better alignment and diverse moire patterns is proposed to empower the network training. The proposed method achieves SOTA scores in the image and video demoireing tasks and promising visual results.", "questions": "The authors should do a thorough comparison between this paper and [30] to justify the novelty and highlight the contributions. Also the authors should properly cite [30] wherever necessary.\n\nHow does the proposed method trained on the dataset generalizes to an unknown combination of the camera, display and distances? It is possible to justify this using the proposed dataset as it has 4 combinations. For example, the model can be trained on 3 combinations and evaluated on the other one. Unfortunately, I cannot tell how the data is split from the paper. The authors are encouraged to clarify this part, and if the split cannot justify the generalization ability, I suggest an extra experiment on this topic.\n\nAlso, how much do distance and viewing angle change? And what is the physical setup of data capture? Do the distance and viewing angle change during the video capture process? A handheld camera usually involves distance and angle changes caused by hand motion, which triggers changes in moire patterns. So it is good to have varying moire patterns in one video. Specifying related details will be helpful for readers to construct their own datasets when necessary.\n\nWhy don't make a video dataset using data synthesis approaches as described in [27, 28]? To my opinion data synthesis can greatly diversify the dataset and avoid alignment issues described in Sec 3.\n", "limitations": "* The dataset involves several combinations of displays and cameras, which may be still limited. A model trained on it may bias towards given cameras and display and not reasonably generalize to unseen combinations, unless the authors prove it as mentioned in the Question section.\n* As the authors mentioned, the proposed method doesn't consider the temporal consistency of the demoireing results, which may cause the jittering issue.\n", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "* The topic is of broad interest as it is not uncommon to use a cellphone camera to capture contents on the display.\n* It is the first paper to explore raw video demoireing. \n* The design of the modulations is reasonable. Cross-channel modules in the network better integrate information from 4 raw channels and large receptive fields in cross-spatial modules can handle large moire patterns.\n* Compared to the existing video demoireing dataset, the constructed dataset has extra raw signals, more diverse moire patterns and better alignment.\n* The proposed method achieves SOTA scores and good visual results.\n", "weaknesses": "* There are existing papers on raw image demoireing [30] and video demoireing [3]. Interestingly, some observations/designs in the paper seem to be consistent with observations/designs in [30] but [30] is not cited in those places, including:\n\t* finding that the moire pattern in the raw domain is simpler\n\t* channel-wise modulation through learnable parameters\n\t* loss function design\nThe above can considerably diminish the novelty of the paper.\n\n* As the statement of the dataset and train/val/test split is not sufficiently clear (specified in the Question section), I’m unsure about the generalization ability of the proposed method.\n* The proposed method doesn't consider the temporal consistency of the demoireing results. This may raise the flickering issue when the method deals with a video with varying moire patterns.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your nice summary and positive comments on our contributions and dataset.  In the following, we give detailed responses to your comments. We only give the outline of your questions due to the limited characters. \n\n> About the comparison with [30]. \n\nThanks!  As you mentioned, [30] pointed out that the moiré pattern in raw domain is simpler and [30] utilized hybrid loss functions. We will cite [30] in those places. Note that, the two points are not our contributions in this work. Our main contribution is based on our key observations that the moiré patterns in different raw channels are different. If we utilize traditional convolution to recover shared structures, the moiré patterns in different channels are also spread. Therefore, we propose to use the complementariness between color-separated features and color-mixed features. To our knowledge, there is no work giving this observation and no work exploring the complementariness of color-separated features and color-mixed features for image/video demoiréing. \n\nSpecifically, we propose a channel-modulation mechanism to modulate the weights of the color-separated features. If we replace the modulation operation by channel attention, which is used in [30], the performance will be degraded, as shown in Table R2. As shown in Fig. R5, the learned modulation weights have a large variance, which implies that different channels contribute very differently. In contrast, the weights generated by channel attention are similar, which cannot give distinctive enhancements to different channels. Therefore, our modulation strategy is better than the channel attention strategy. We will give a thorough comparison between this work and [30] in our final version. \n\nTable R2: Comparison between our channel modulation and traditional channel attention. \n|Methods|PSNR|SSIM|LPIPS|\n|----|----|----|----|\n|channel attention|28.492|0.9170|0.0987|\n|channel modulation|**28.706**|**0.9201**|**0.0904**|\n\n> About the generalization ability of the proposed method.\n\nThanks! For video demoiréing, both our training and test datasets include the four device combinations. For each combination, the train and test splits are about 84%/16%. Fortunately, the image demoiréing is conducted on the dataset provided by [30], whose test set contains two device combinations that are not included in the training set. Therefore, we utilize these test images (about 1/4 of the whole test set) to evaluate the generalization ability of different models. As shown in Table R5, our method still achieves the best results in terms of SSIM and LPIPS, and the PSNR value is only slightly inferior to UHDM*. Note that, for our method, if the test camera-screen combination exists in the training set, the color tone of the test images can be well recovered. Otherwise, the recovered image may have color cast. The main reason is that our network also performs the ISP process, and the ISP process is indeed different for different cameras. Fortunately, the moiré patterns can still be removed. Therefore, our LPIPS value is still small. In summary, our method has the best generalization ability in terms of moiré removal. \n\nTable R5: Comparison of the generalization ability of different models by evaluating with the screen-camera combinations that are not included in the training set. \n|Methods|PSNR|SSIM|LPIPS|\n|----|----|----|----|\n|$FHDe^{2}Net$|22.8401|0.8968|0.1530|\n|RDNet|24.7979|0.9280|0.0846|\n|MBCNN|24.7264|0.9253|0.1017|\n|MBCNN*|24.6585|0.9314|0.0925|\n|UHDM|24.6789|0.9239|0.0955|\n|UHDM*|**25.7693**|0.9310|0.0759|\n|Ours|25.7187|**0.9333**|**0.0727**|\n\n>About the dataset capturing details. \n\nThanks! As you mentioned, using a handheld camera can result in changes in moiré patterns. We intentionally utilized handheld devices to capture videos for the purpose of obtaining varying moiré patterns. In order to capture the videos clearly, we change the distances between our camera and the display screen from 10 to 25 centimeters. The view angle changes from 0 to 30 degrees. Our captured moiré patterns have a large variance in terms of scales, colors, and stripes, as shown in Fig. 2 of the supplementary file. \n\n>Why not using data synthesis approaches [27, 28]? \n\nWe would like to point out that there is a significant gap between the synthesized moiré and real moiré patterns. Fig. R6 provides a comparison between [27, 28] and our real dataset. For CFAMoiré [27], it cannot generate moiré patterns in flat areas. However, for recaptured screen images, the flat areas also contain heavy moiré patterns. For LCDMoiré, although it has moiré patterns in flat areas, it cannot synthesize multi-scale interleaving moiré patterns. As shown in Fig. R6, various moiré pattern cycles can be observed on the same image for real captured ones. However, the synthesized moiré patterns lack this feature. In addition, real captured moiré patterns often have diverse colors and morphologies, which are relatively limited in synthesized moiré patterns. Finally, current moiré synthesis methods are all designed for sRGB domain moiré synthesis other than raw domain moiré synthesis. As shown in Fig. R4, even utilizing inverseISP to convert real moiré images into raw domain, we still cannot generate the raw domain moiré patterns that are close to real ones. Therefore, in this work, we captured a raw video demoiréing dataset and generate space and time well aligned pairs to benefit the training and evaluation process. \n\n>About temporal consistency.\n\nOur video demo shows the video moiré removal results and there are slightly jittering artifacts. To solve this problem, we further fine-tune our network by introducing temporal loss [3]. The demoiréing performance and temporal consistence are further improved (see Table R6). We will present the video results in our website.\n\nTable R6:  Ablation on temporal loss.\n|Methods|PSNR|SSIM|LPIPS|\n|----|----|----|----|\n|w/o temporal loss|28.706|0.9201|0.0904|\n|w temporal loss|28.968|0.9200|0.0884| \n"}]

Author rebuttal:
Dear Reviewers,

We sincerely thank you all for your careful review and helpful suggestions, which significantly helped to improve our paper. In the following, we give separate responses to your comments one by one. The figures used for further illustration are presented in the pdf file, and they are named as Figures. R1-R6. 

Best Regards,

Authors of 6552

Meta-review: This paper presents a new approach for video demoireing. It observes that different channels have different moire patterns, and proposes an approach that leverages this fact. Reviewers also appreciated that it introduces a new video dataset for the task. While there was some concern about limited novelty, in general the reviewers agreed that the solution was validated by experimental results, and that this work may inspire future research in this field.
Decision: Accept (poster)