R EAL -T IME
I MAGE
D EMOIR ´ EING
ON
M OBILE
D EVICES
Anonymous authors
Paper under double-blind review
A BSTRACT
Moir´e patterns appear frequently when taking photos of digital screens, drastically degrading the image quality.
Despite the advance of CNNs in image demoir´eing, existing networks are with heavy design, causing redundant computation burden for mobile devices.
In this paper, we launch the first study on accelerating demoir´eing networks and propose a dynamic demoir´eing acceleration method (DDA) towards a real-time deployment on mobile devices.
Our stimulus stems from a simple-yet-universal fact that moir´e patterns often unbalancedly distribute across an image.
Consequently, excessive computation is wasted upon non-moir ´ e areas.
Therefore, we reallocate computation costs in proportion to the complexity of image patches.
In order to achieve this aim, we measure the complexity of an image patch by designing a novel moir ´ e prior that considers both colorfulness and frequency information of moir ´ e patterns.
Then, we restore image patches with higher-complexity using larger networks and the ones with lower-complexity are assigned with smaller networks to relieve the computation burden.
At last, we train all networks in a parameter-shared supernet paradigm to avoid additional parameter burden.
Extensive experiments on several benchmarks demonstrate the efficacy of our proposed DDA.
In addition, the acceleration evaluated on the VIVO
X80 Pro smartphone equipped with a chip of Snapdragon 8 Gen 1 shows that our method can drastically reduce the inference time, leading to a real-time image demoir ´ eing on mobile devices.
Source codes and models are anonymously released in the supplementary materials.
I NTRODUCTION Moir´e
patterns (Sun et al., 2018; Yang et al., 2017b) describe an artifact of images that in particular appears in television and digital photography.
In contemporary society, using mobile phones to take screen pictures has become one of the most productive ways to record information quickly.
Nevertheless, moir ´ e patterns occur frequently from the interference between the color filter array (CFA) of camera and high-frequency repetitive signal.
The resulting stripes of different colors and frequencies on the captured photos drastically degrade the visual quality.
Therefore, developing demoir´eing algorithms has received long-time attention in the research community, and yet remains unsolved in particular when running algorithms on mobile devices.
Primitive studies on image demoir´eing resort to traditional machine learning techniques such as low-rank and sparse matrix decomposition (Liu et al., 2015) and bandpass filters (Yang et al., 2017a).
The rising of convolutional neural networks (CNNs) has vastly boosted the efficacy of image demoir´eing (He et al., 2019; Zheng et al., 2020).
However, the improved quantitative performance, such as PSNR (Peak Signal-to-Noise Ratio), comes at the increasing costs of energy and computation.
For example, MBCNN (Zheng et al., 2020) eats up 4.22T floating-point operations (FLOPs) in order to restore a 1920 × 1080 smartphone-taken moir´e image.
Given that the moir´e patterns mostly emerge in mobile photography, such massive computations carry considerable inference latency, preventing a real-time demoir´eing for ease of the users.
Such handicap could be more serious when it comes to video demoir´eing.
Therefore, it is of great need to bridge the technology gap between academia and industry.
To tackle the aforementioned issue, we initiate the first study on accelerating demoir´eing networks towards a real-time deployment on mobile devices.
Our motivation arises from empirical actuality in moir ´ e images.
As shown in Fig.
1, an image is often partially contaminated by the moir ´ e pattern.
Some areas are filled with intensive moir ´ e stripes, some are much relieved while some are kept away from moir ´ e pollution.
It is natural to hand out computation to these moir ´ e centralized areas but less to these diluted areas.
In the extreme case, it is needless to cleanse uninfluenced areas.
Unfortunately, existing methods Sun et al.
; Zheng et al.
(2020) have not distinguished the treatment to the different areas in an image.
They not only waste excessive computation on non-moir´e areas but also bring about side effect, such as over-whitened image contents.
Therefore, reallocating computation costs in compliance with the complexity of a moir ´ e area can be a potential solution to accomplish real-time image demoir ´ eing on mobile devices.
Stimulated by the above analysis, we opt to split a whole image into several sub-image patches.
To measure the patch moir ´ e complexity, we introduce a novel moir ´ e prior.
As can be referred in Fig.
1, moir´e patterns are featured with either high frequency or rich color information.
Thus, we define the moir´e prior as the product of frequency and color information in a patch.
In detail, we model the frequency information by a Gaussian filter and the colorfulness metric is a linear combination of the mean and standard deviation of the pixel cloud in the RGB colour space (Hasler & Suesstrunk, 2003).
Using this prior to measure the moir ´ e complexity, each image patch is then processed by a unique network with its computation costs in proportion to the moir ´ e complexity.
In this fashion, larger networks are utilized to restore moir ´ e centralized areas to ensure the recovery quality while smaller networks are leveraged to restore moir ´
e diluted areas to relieve computation burden.
Consequently, we lead to a better tradeoff between the image quality and resource requirement on mobile devices.
Nevertheless, multiple networks lead to more parameter burden, which also causes deployment pressure due to the short-supply memory on mobile devices.
To mitigate this issue, we leverage the supernet paradigm (Yang et al., 2021) to jointly train all networks in a parameter-shared manner.
Concretely, we regard the vanilla demoir´eing network as a supernet, and weight-shared subnets of different sizes are directly extracted from this supernet to process image patches of different demoir´e complexity.
During the training phase, each subnet is dynamically trained using corresponding image patches with similar moir´e complexity.
Consequently, the overall running overhead can be effectively reduced without introduction of any additional parameters.
We have conducted extensive experiments for accelerating existing demoir´eing networks on the
LCDMoir´e (Yuan et al., 2019) and FHDMi (He et al., 2020) benchmarks.
The results show that our dynamic demoir´eing acceleration method, termed DDA, achieves an obvious FLOPs reduction even with PSNR and SSIM increases.
For instance, DDA reduces 45.2% FLOPs of the state-of-the-art demoir´eing network MBCNN Zheng et al.
(2020) with 0.35 dB PNSR increase.
Furthermore, the accelerated DMCNN Huang et al.
(2017) leads to real-time image demoir´eing on the VIVO
X80 Pro smartphone, with a tiny latency of 48.0 ms when processing an image of 1920 × 1080 resolution.
This work addresses the problem of demoir´eing network deployment on resource-limited mobile devices.
The key contributions of this paper include: (1) A novel framework for accelerating image
[Figure]
demoir´eing networks in a dynamic manner.
(2) An effective moir´e prior to identify the demoir´eing complexity of a given image patch.
(3) Performance maintenance and apparent acceleration on modern smartphones.
R ELATED W ORK 2. I MAGE D EMOIR ´ EING
Image demoir ´ eing aims at removing moir ´ e patterns on captured images.
Early work mainly focuses on manually designed algorithms with the aid of low-rank & sparse matrix decomposition (Yang et al., 2017a;
Liu et al., 2015).
With the explosion and popularity of deep learning, extensive demoir´eing networks are proposed in recent years to achieve moir´e removal in an end-to-end manner (Sun et al., 2018;
He et al., 2019).
As a pioneering work, (Sun et al., 2018) proposed a multiscale network structure to remove moir ´ e patterns at different frequencies and scales.
(He et al., 2019) dived into designing specific learning schemes to resolve the unique properties of moir ´ e patterns including frequency distributions, edge information and appearance attributes.
(Zheng et al., 2020) reformulated the image demoir ´ eing problem as moir ´ e texture removal and color restoration, and proposed MBCNN (Zheng et al., 2020) which consists of a learnable bandpass filter to learn the frequency prior and a two-step tone mapping mechanism to restore color information.
FHDe2Net (He et al., 2020) uses a global branch to eradicate multi-scale moir ´ e patterns and a local branch to reserve fine details.
(Liu et al., 2020) further proposed WavleNet to handle demoir´eing in the wavelet domain.
The same dilemma for the aforementioned networks is their huge computation burden, which greatly prohibits the practical deployment on mobile devices.
2. D YNAMIC N ETWORKS AND S UPERNETS
Dynamic networks adapt the network structures or parameters w.r.t.
different inputs (Kong et al., 2021; Huang et al., 2017; Bolukbasi et al., 2017).
Due to the advantages in accuracy performance and computation efficiency, dynamic networks have received increasing research interest in recent years.
A comprehensive overview of dynamic networks can be found at (Han et al., 2021).
Supernets are a type of dynamic network that reserves weight-shared sub-networks of multiple sizes in one network, and randomly samples these sub-networks for training (Chen et al., 2022;
Yang et al., 2021; Yu & Huang, 2019).
According to the resource constraints, different sub-networks with varying widths and resolutions can be chosen during the testing phase without additional parameter burden.
Inspired by these studies, our proposed DDA involves the supernet paradigm by dynamically allocating image patches with different demoir´eing complexity to their corresponding sub-networks.
M ETHODOLOGY
The general framework of our proposed dynamic demoir´eing acceleration method (DDA) is manifested in Fig.
A moir´e image is firstly split into several sub-image patches, which are then reorganized into different groups in conformity with the patch moir´e complexity.
This is achieved
[Figure]
by a novel moir´e prior that considers both color and frequency information of moir´e patterns.
We detail it in Sec. 3.1.
For the purpose of real-time image demoir´eing on mobile devices, we train multiple networks of different complexity to process patches in different groups.
In the training and testing phases, image patches with higher-complexity are fed to larger networks and patches with lower-complexity are dealt with smaller networks.
Sec. 3.2 gives the implementations.
Finally, as an alternative for training separate networks that result in more parameters, we regard each one as a subnet of the vanilla demoir´eing network (supernet), leading to a weight-shared training paradigm as depicted in Sec. 3.3.
3. M OIR ´ E P RIOR
The complexity degree of moir´e patterns can be determined by a human, however, it is lavish and laborious to manually define the complexity for all patches in every single one of the moir´e image.
Many former works on dynamic networks (Han et al., 2021; Kong et al., 2021) choose to train an additional module to adapt the network w.r.t.
different inputs, which however, brings unexpected parameters and computations for its compositions of several convolutional or fully-connected layers.
Given our plan of deploying demoir ´ eing networks on mobile devices, such a solution is not feasible, or at least not optimal.
To solve this problem, we propose a novel moir´e prior, which opens a new road to measure the moir ´ e complexity of a given moir ´ e image in a fast manner.
The motivation for this prior comes from an in-depth observation on moir ´ e images.
As can be inferred from Fig.
3, moir ´ e patterns vary a lot in frequency and colorfulness.
Customarily, a perceptible moir ´ e pattern is highlighted by either high frequency or rich color information.
Therefore, a prior reflecting both image frequency and colorfulness can be an efficacious method to model the intensity of moir ´ e patterns.
Denoting a moir ´ e image as X , we first decompose it into sub-image patches as { x i } Ni =1 .
For a specific patch x , we use the Gaussian high-pass filter (Dogra & Bhalla, 2014) with a standard deviation of 5 for the Gaussian distribution to extract the frequency information as F ( x ) .
To measure the patch colorfulness, we consider a linear combination of the mean and standard deviation of the pixel cloud in the RGB colour space (Hasler & Suesstrunk, 2003):
[Equation]
where µ ( · ) and σ ( · ) are the mean and standard deviation functions, x R , x G , x B denote the R, G, B color channels, respectively.
Here 0 .
3 is a parameter found by (Hasler & Suesstrunk, 2003) through maximising the correlation between the experimental data and the colorfuness metric.
Refer to (Hasler & Suesstrunk, 2003) for more principles of measuring image colorfulness.
Therefore, our proposed moir´e complexity score using frequency and colorfulness prior is finally defined as:
[Equation]
where µ ( · ) is the mean function.
Fig.
3 shows that M ( x ) can be a reliable metric for evaluating the patch moir´e complexity.
Notice that, without building any extra network module, the operations of our moir´e prior become highly cheap, bringing negligible computation burden.
3. D YNAMIC D EMOIR ´ EING A CCELERATION
In image demoir´eing, an moir´e-polluted image X is expected to restore to moir´e-free ground-truth in natural scenes.
A traditional demoir´eing process is formulated using CNNs as:
[Equation]
where F standards for the demoir´eing network with its parameters denoted as Θ .
As can be seen from Eq.
(3), existing methods restore all areas of a moir´e image with the same network F , which wastes excessive computation since the moir´e complexity varies significantly across different areas of an image as aforementioned in Sec. 1.
This violates our goal of real-time image demoir´eing on mobile devices.
A natural way for demoir´eing acceleration is to reallocate computation costs according to the complexity of a moir´e area.
To that effect, we reorganize the sub-image patches { x i } Ni =1 of X in an ascending order of their moir´e complexity score defined in Eq.
(2), and then split the ordered patches into M groups denoted as { G 1 , G 2 , ..., G M } .
Each group G i contains these image patches, moir´e complexity scores of which range from the top- (cid:0) ( i − 1) ·⌈ NM ⌉ +1 (cid:1) to - (cid:0) i ·⌈ NM ⌉ (cid:1) smallest among all.
Then, we construct M different demoir´eing networks {F i } Mi =1 with parameters of different sizes as { Θ i } Mi =1 , and process each image patch x ∈ G
i using the i -th network F i as:
[Equation]
x
[Equation] [Equation]
In our setting, the complexity of F i is smaller than that of F i +1 such that smaller-complex image patches can be handled by networks with low computation costs , and vice versa.
Eq.
(4) dynamically accelerates the derivation of Eq.
(3) by assigning computation costs in line with the degree of moir ´ e complexity.
Meanwhile, the recovery quality is still ensured as moir ´ e centralized areas are restored using larger networks.
Finally, the moir ´ e-free output of our dynamic demoir ´ eing acceleration method (DDA) is obtained by concatenating patch outputs of all networks:
[Equation]
where concat () concatenates the output patches to construct a moir ´ e-free full image.
3. S UPERNET T RAINING
Though the aforesaid procedure benefits reduction of the overall computation costs, the challenge arises in respect of parameter burden when deploying our demoir ´ eing method on mobile devices featured with short-supply memory.
For a simple case, setting the largest network F M as the vanilla demoir ´ eing network F , additional parameters of (cid:80) M − 1 i =1 ( | Θ i | ) are introduced in total.
To solve this, in place of training networks { F i } Mi =1 in isolation, we further propose to use the supernet paradigm (Yang et al., 2021) to train and infer all networks in a parameter-shared manner.
In detail, the vanilla demoir ´ eing network F is regarded as a supernet and its parameters Θ are partly shared by F i .
Supposing the network width of F i is W i , we inherit the first W i proportion of convolution filter weights to the subnet
F i , denoted as Θ
[ W i ] .
Consequently, the network F i becomes a subnet of F .
Therefore, our moir ´ e-free output is finally reformulated as:
[Equation]
As a consequence, image demoir ´ eing can be effectively accelerated even without any additional parameter burden, which finally reaches our target for a real-time deployment on mobile devices.
E XPERIMENT 4. E XPERIMENT S ETUP 4.1. D ATASETS
There are three main public datasets for image demoir´eing.
(1) LCDMoir´e dataset Yuan et al.
from the AIM19 image demoir´eing challenge consists of 10,200 synthetically generated image pairs including 10,000 training images, 100 validation images and 100 testing images at 1024 ×resolution.
(2) FHDMi dataset (He et al., 2020) contains 9981 image pairs for training and 2019 for testing with 1920 × 1080 resolution.
The TIP2018 dataset (Huang et al., 2017) consists of real photographs constructed by photographing images with 400 × 400 resolution from ImageNet (Deng et al., 2009) displayed on computer screens.
In this paper, we conduct experiments on the LCDMoir´e and FHDMi datasets.
We do not consider the TIP2018 benchmark since the resolution is too small to meet our target for image demoir´eing on mobile devices as the captured images generally have an extremely high-resolution with 1920 × 1080 or higher.
[Table]
E VALUATION P ROTOCOLS
We adopt the widely-used metrics of PSNR (Peak Signal-to-Noise Ratio) and SSIM (Structure Similarity) to conduct a quantitative comparison for demoir´eing performance.
We also report the color difference between restored images and the clean images using the CIE DeltaE 2000 (Sharma et al., 2005) measurement, denoted as ∆ E. The Float Points Operations (denoted as FLOPs) and network latency per image on VIVO X80 Pro smartphone with the Snapdragon 8 Gen 1 chip of networks are reported as the accelerating evaluation.
4.1. B ASELINES
We choose to accelerate DMCNN (Huang et al., 2017) and MBCNN (Zheng et al., 2020) to verify the efficacy of our DDA method.
DMCNN is a pioneering network for image demoir ´ eing with multiscale structure.
MBCNN is a state-of-the-art demoir ´ eing network, which consists of a learnable bandpass filter to learn the frequency prior and a two-step tone mapping mechanism to restore color information.
We report the results of baseline models as well as their compact versions by slimming the network width based on our re-implementation on the PyTorch framework (Paszke et al., 2019).
4.1. I MPLEMENT D ETAILS
Our implementation of DDA is based on the PyTorch framework (Paszke et al., 2019), with the group number M = 3 , width list W = { 0 . 25 , 0 . 5 , 0 . 75 } on FHDMi and W = { 0 . 4 , 0 . 5 , 0 . 6 } on
LCDMoir ´ e. We split the original images of LCDMoir ´ e and FHDMi datasets into sub-image patches of 512 × 512 and 640 × 540, respectively.
Then, we classify the sub-image patches into multiple groups with different moir ´ e complexity using our proposed moir ´ e prior.
We train the supernet using
Adam (Kingma & Ba, 2014) optimizer.
The initial learning rate and batch size are set to 1e-4 and 4 in all experiments.
During training, we iteratively extract a batch of image pairs within a specific class of moir ´ e complexity, which is trained using the subnet of corresponding width extracted from the supernet.
For DMCNN, we give 200 epochs for training with the learning rate divided byat the 100-th epoch and 150-th epoch.
For MBCNN, we follow (Zheng et al., 2020) to reduce the learning rate by half if the decrease in the validation loss is lower than 0.001 dB for four consecutive epochs and stop training once the learning rate becomes lower than 1e-6.
All experiments are runned on NVIDIA Tesla V100 GPUs.
4. P ERFORMANCE A NALYSIS
In this section, we perform detailed performance anon the different components of our DDA including the supernet training paradigm and moir´e prior.
4.2. S UPERNET TRAINING
We first conduct experiments to investigate how the hyper-parameters in the supernet training influence the performance of DDA, w.r.t , the width list configuration W .
The experiments are conducted on the FHDMi He et al.
and LCDMoir ´ e Yuan et al.
(2019) datasets using MBCNN.
We can observe from Tab. 1 that a dispersed configuration W = { 0 . 25 , 0 . 5 , 0 .
75 } performs better on the FHDMi dataset, while a compact configuration W = { 0 . 4 , 0 . 5 , 0 . 6 } works better on the LCDMoir´e dataset.
To explain, LCDMoir´e is a synthetic dataset, where the moir´e patterns distribute more balanced compared with FHDMi that is captured using embedded cameras.
Self-confidently, a more discrete width list guarantees our purpose of dynamically removing moir´e patches of different complexity in real scenarios.
[Table] [Table]
[Table] [Table]
[Table]
Besides, Tab. 2 compares the performance for accelerating MBCNN on LCDMoire between supernet training and respectively training each sub-network.
It can be seen that supernet training does not lead to performance degradation and it drastically reduces the parameter burden compared with simultaneously keeping multiple sub-networks, which demonstrates the effectiveness of our DDA for practical deployment.
4.2. M OIR ´ E PRIOR
We further analyze the performance of the proposed moir ´ e prior.
The experiments are conducted on the FHDMi dataset using MBCNN.
We use networks of different widths to infer different groups of pictures with different complexity classified by our proposed moir ´ e prior.
The results for Tab.show that all widths perform similarly for the easiest group, while larger width significant outperforms smaller width for the group with highest moir ´ e complexity.
Such results demonstrate the effectiveness of our proposed moir ´ e prior operator, and also validate our point that using large networks to restore patches with low moir ´
e complexity wastes massive computation resources.
At last, we investigate three variants of our proposed moir ´ e prior including (1) only using highfrequency information (denoted as F) (2) only using colorfulness information (denoted as C) (3) adding two information scores instead of multiplication in Eq.
(2) (denoted as C + F).
As shown in Tab. 4, all variants result in worse performance, which well demonstrates the efficacy of our proposed moir ´ e prior that considers both color and frequency properties of moir ´ e patterns.
It is worth mentioning that C+F implies domination of the colorfulness measurement due to the fact that scores given by C are generally two orders of magnitude larger than that of F in our observation.
As a result, C+F leads to a similar performance to C. In contrast, by multiplying both scores, our prior offers reliable moir´e complexity for a given image patch.
4. Q UANTITATIVE C OMPARISON
Tab. 5 and Tab.
6 report the quantitative results of our DDA for accelerating DMCNN and MBCNN.
On FHDMi, DDA surprisingly improves the PSNR of MBCNN by 0.35 dB even with a FLOPs reduction of 45.2%.
We attribute such results to that the original baseline assigns the same network to restore the areas with little or no moir´e patterns, which may damage the original details of the image.
Consequently, the poor performance barricades the usage of demoir ´ eing networks in practical deployment.
In contrast, DDA leverage the smallest network to restore these non-moir´e areas, leading to a better global demoir´eing effect.
Besides, we demonstrate the effectiveness of DDA by comparing MBCNN accelerated by it with several state-of-the-art demoir´eing networks including
MDDM (Cheng et al., 2019), MopNet (He et al., 2019), FHDe 2 Net (He et al., 2020) on FHDMi dataset.
As can be seen from Tab. 7, DDA can outperform other networks regarding both complexity reduction and demoir´eing performance.
For instance, DDA surpasses FHDe 2 Net by 0.69 dB
[Table] [Table]
[Table] [Table]
[Table]
PSNR with even far fewer FLOPs (2.13 T for DDA and 11.41 T for FHDe 2 Net), which shows the effectiveness of our perspective for reallocating computation costs in proportion to the complexity of moir ´ e patches.
As to LCDMoir ´ e
[Table]
compared with DMCNN-0.75 which simply infers the whole image, our DDA dynamically assigns computation resources with respect to moir ´ e complexity of patches, retaining a better PSNR performance (34.19 dB for DDA and 33.41 dB for DMCNN-0.75) and more
FLOPs reduction (55.1% for DDA and 31.4% for DMCNN-0.75).
Meanwhile, DDA achieves a noticeable latency reduction of 56.9 ms for accelerating MBCNN (75.2ms for DDA and 132.1ms for the baseline), enabling a real-time image demoir ´ eing on mobile devices.
Nevertheless, a noticeable performance drop of PSNR is still observed (41.68 dB for DDA and 43.95 dB for the full model, and comparison results with state-of-the-art networks including MDDM (Cheng et al., 2019),
MDDM+ (Cheng et al., 2021) and MopNet (He et al., 2019) in Tab. 8 also suggest a relatively poor result of DDA than its performance on FHDMi.
Here we argue that the LCDMoir´e dataset is built on simulating the aliasing between CFA and the screen’s LCD subpixel, which results in images with different distribution of moir´e patterns compared with smartphone-captured moir´e images.
Compared with smartphone-captured FHDMi with different moir´e distributions, the moir´e patterns in
LCDMoir´e are much more uniform and their cropped moir´e patches are of similar complexity.
This explains the relatively poor performance of DDA on the LCDMoir´e dataset.
Note that our approach even improves the performance of baseline models with less computation burden on FHDMi and achieves superior performance in comparison with SOTA networks.
Given our motivation for the practical deployment of image demoir´eing in real cases, the efficacy of the proposed method is still affirmative.
4. Q UALITATIVE C OMPARISON
In addition to the quantitative results, Fig.
4 further displays the visualization results of restored images on the FHDMi dataset.
Results on the LCDemoir´e dataset can be found in Appendix A.2.
As can be observed, uniformly performing the same accelerating rate for the whole image (MBCNN-
[Figure]
0.5 × ) drastically degrade the performance as the areas with dense moir ´
e patterns do not receive enough computation resource to be efficiently restored.
In contrast, by dynamically allocating the computational resources, our DDA achieves promising demoir ´ eing quality compared with the original network.
Thus, the efficacy of DDA for accelerating demoir ´ eing networks for practical application is well demonstrated.
L IMITATION AND F UTURE WORK
We further discuss the limitations of our DDA, which will be our future focus.
Firstly, DDA simply divides all images into equal-number patches in each class, laying some avenues for future research in devising image-aware classification priors.
Besides, our limited computing facilities prevent us from accelerating other demoir ´ eing networks with varying structures (Liu et al., 2020;
He et al., 2019).
More validations are expected to be performed to further demonstrate the efficacy of DDA.
C ONCLUSION
In this paper, we present a novel dynamic demoir´eing acceleration method (DDA) to reduce the huge computational burden of existing networks towards real-time demoir´eing on mobile devices.
Our
DDA is based on the observation that the moir´e complexity is highly unbalanced across different areas of an image.
On this basis, we propose to split the whole image into sub-patches, which are then regrouped according to their moir´e complexities measured by a novel moir´e prior that considers both the frequency and colorfulness information.
Then, we use models with different sizes to restore patches in each group.
In particular, larger networks are utilized to restore moir´e centralized areas to ensure the recovery quality while smaller networks are leveraged to restore moir´e diluted areas to relieve computation burden.
To avoid the additional parameter burden caused by retaining multiple networks, we further leverage the supernet paradigm to jointly train the networks in a parametershared manner.
Results on several benchmarks demonstrate that our method can effectively reduce the computation costs of existing networks with negligible performance degradation, enabling a real-time demoir´eing on current smartphones.
A. Q UALITATIVE R ESULTS ON THE LCDM OIR
[Figure]
E DATASET
[Figure]