Multi-Scale Adaptive Network for Single Image Denoising
Abstract
Multi-scale architectures have shown effectiveness in a variety of tasks thanks to appealing cross-scale complementarity.
However, existing architectures treat different scale features equally without considering the scale-speciﬁc characteristics, i.e. , the within-scale characteristics are ignored in the architecture design.
In this paper, we reveal this missing piece for multi-scale architecture design and accordingly propose a novel Multi-Scale Adaptive Network (MSANet) for single image denoising.
Speciﬁcally, MSANet simultaneously embraces the within-scale characteristics and the cross-scale complementarity thanks to three novel neural blocks, i.e. , adaptive feature block (AFeB), adaptive multi-scale block (AMB), and adaptive fusion block (AFuB).
In brief, AFeB is designed to adaptively preserve image details and ﬁlter noises, which is highly expected for the features with mixed details and noises.
AMB could enlarge the receptive ﬁeld and aggregate the multi-scale information, which meets the need of contextually informative features.
AFuB devotes to adaptively sampling and transferring the features from one scale to another scale, which fuses the multi-scale features with varying characteristics from coarse to ﬁne.
Extensive experiments on both three real and six synthetic noisy image datasets show the superiority of MSANet compared with 12 methods.
The code could be accessed from https://github.com/XLearning-SCU/2022-NeurIPS-MSANet.
Introduction
In the real world, images are often contaminated by various signal-dependent or -independent noises during the image acquisition process.
As a result, the imaging quality will deteriorate, thus hindering people and computers from receiving image information.
To solve this problem, image denoising, as an essential step for image perception, has been extensively studied in the past decades [10, 19, 27, 50].
In the early, ﬁltering-based methods remove the image noise by manually designing low-pass ﬁlters, e.g. , median ﬁltering [8], bilateral ﬁltering [36], and wiener ﬁltering [7].
Afterward, model-based methods remove the image noise by optimizing a problem of maximum a posteriori [13, 40, 43].
For instance, ITS [40] proposed an intrinsic tensor sparsity regularization on the non-local similar image patches by assuming they could be sparsely represented.
Looking from the other side, both the ﬁltering- and model-based methods are based on image priors from the statistics of natural images, and thus could be referred to as prior-based methods.
Although remarkable performance has been achieved, an unpleasant denoising result will be obtained once the priors are inconsistent with the real data distribution.
To avoid prior engineering, learning-based methods adopt a data-driven fashion to remove noise by learning the mapping from the noisy image to the corresponding clean image.
[Figure]
G. Det.
N. Rob.
N. Amt.
F. Chs.
R. Fie.
F. Res.
C. Inf.
With the popularity of deep neural networks, various network architectures have been designed and achieved state-of-the-art performance [6, 26, 32, 51].
For example, DnCNN [50] introduced residual learning and batch normalization to implement a denoising convolution network.
SwinIR [19] introduced the shifted windowing scheme to implement a restoration Transformer network.
Among the network design paradigms, multi-scale architectures play a signiﬁcant role in performance improvement thanks to multi-scale features.
However, existing studies design their network architectures by only considering the cross-scale complementarity while ignoring the within-scale characteristics (see Fig. 1).
To be exact, the shallow and high-resolution features, which lack awareness of contextual information, are sensitive to inputs and contain a lot of noises.
Nevertheless, they contain abundant image geometric information, such as edges and textures.
In consequence, they are critical for the recovery of ﬁne-grained image details, and their network structures should take the advantages and make up for the disadvantages.
The deep and low-resolution features are competitive in noise-robustness and contextual information, which are critical for the recovery of coarse-grained image context, but the too-low resolution will destroy the image structure.
Hence, their network structures should also make full consideration of their characteristics during the architecture design.
To summarize, as different scale features show varying characteristics, it is deserved to deal with them via scale-speciﬁc structures to fully adapt to their characteristics.
Based on the motivation, we propose a novel Multi-Scale Adaptive Network (MSANet) for single image denoising, which simultaneously incorporates the within-scale characteristics and the crossscale complementarity into architecture design by overcoming the following three difﬁculties, i.e. , i) how to adaptively sample image details and ﬁlter noises; ii) how to adaptively extract contextually informative features without changing feature resolutions; iii) how to adaptively fuse the multi-scale features with varying characteristics.
Accordingly, three neural blocks, i.e. , adaptive feature block (AFeB), adaptive multi-scale block (AMB) and adaptive fusion block (AFuB) are proposed.
In brief, AFeB handles the features with mixed details and noises through adaptively sampling and weighting, and thus the image details are preserved while ﬁltering noises.
AMB extracts the contextual information through dilated convolutions and adaptive aggregation, and thus contextually informative features are obtained while keeping the resolution unchanged.
AFuB adaptively samples and transfers the features from one scale to another scale, and the multi-scale features with varying characteristics are fused from coarse to ﬁne.
To summarize, the contributions are as follows:
• We propose a novel neural network for single image denoising, termed as MSANet.
The major difference with existing methods is that MSANet simultaneously considers and incorporates the within-scale characteristics and the cross-scale complementarity into multiscale architecture design, which is a missing piece before and the ﬁrst revelation so far.
• To exploit the within-scale characteristics and achieve the cross-scale complementarity, we design three neural blocks, i.e. , AFeB, AMB, and AFuB, which
are used to implement our
idea, i.e. , building scale-speciﬁc subnetworks corresponding to different scale features by considering their characteristics.
• Extensive experiments are conducted on three real and six synthetic noisy image datasets to show the effectiveness of MSANet, and the signiﬁcance of the within-scale characteristics.
Related Work
In this section, we brieﬂy introduce existing single image denoising methods and multi-scale architectures, and discuss the major differences between MSANet and them.
2.1 Single Image Denoising
In general, most existing single image denoising approaches could be categorized as prior- and learning-based methods.
Prior-based methods are based on some priors from natural images, such as local smoothing [41], self-similarity [36, 10, 13], and sparsity [43, 40].
For instance, BM3D [10] eliminates noisy pixels by transforming the 3D stack of non-local similar patches and employing a shrinkage function to obtain sparse coefﬁcients.
WNNM [13] introduced a low-rank weight coefﬁcient based on the nuclear norm minimization and exploited the non-local similar image patches to remove noise.
Different from prior-based methods that heavily rely on handcrafted priors, learning-based methods learn the mapping from the noisy image to the clean image in an end-to-end manner.
In recent, a large number of methods have been proposed and achieved state-of-the-art performance [6, 14, 32, 38, 47].
For example, MemNet [35] proposed a persistent memory network to fuse both short- and long-term memories for capturing different levels of information.
FFDNet [51] enhanced the denoising network for non-uniform noise by using the noise level map.
Non-local attention [37] was designed to exploit the image self-similarity, and NLRN [20] incorporated it into a recurrent neural network.
SADNet [6] proposed residual spatial-adaptive block and multi-scale context block to constitute a denoising network.
DeamNet [32] introduced an adaptive consistency prior and designed an interpretable deep denoising network.
With the rise of vision Transformers,
Uformer [38] proposed a U-shape Transformer based on the locally-enhanced window Transformer block and multi-scale restoration modulator.
Restormer [47] introduced the channel-based selfattention and gating mechanism to implement an efﬁcient Transformer.
Different from the aforementioned methods, MSANet proposed three neural blocks, i.e. , AFeB,
AMB, and AFuB, by simultaneously considering the within-scale characteristics and the cross-scale complementarity, to constitute the scale-speciﬁc subnetworks corresponding to different scale features for adapting their varying characteristics.
2.2 Multi-Scale Architecture
Multi-scale architectures have played a signiﬁcant role in many ﬁelds of computer vision [5, 17, 21, 48], thanks to multi-scale features and their cross-scale complementarity.
The straightforward way for multi-scale architecture is to separately feed multi-/single-resolution images/features into single/multiple subnetworks, and then fuse the outputs as a result [18, 25, 39, 44, 45].
For example,
HRNet [34] proposed a multi-scale network by gradually adding high-to-low resolution subnetworks and repeating multi-scale fusions for human pose estimation.
CLEARER [12] proposed a multi-scale neural architecture search to automatically determine where to fuse multi-scale features for image restoration.
GDN [22] exploited the multi-scale information using a grid-like network and employed an attention mechanism to improve the performance.
DID-MDN [49] proposed a multi-stream densely connected network to efﬁciently leverage features of different scales for image deraining.
MSCNN [33] consists of a coarse-scale network and a ﬁne-scale network to learn a transmission map for image dehazing.
PANet [25] proposed a pyramid attention network for image restoration by capturing long-range feature correspondences from a multi-scale feature pyramid.
In addition, [6, 11, 38, 47] employed encoder-decoder architecture to combine the high-to-low with low-to-high resolution features through skip-connections.
Although the aforementioned studies and our work share similarities in multi-scale architecture, they are remarkably different.
The existing methods only consider the cross-scale complementarity and use homologous architectures for different scale features.
In contrast, our work additionally considers
[Figure]
TransposedConv2d
[Figure]
the within-scale characteristics and designs scale-speciﬁc structures to embrace both, which is the missing piece and the ﬁrst revelation for multi-scale architecture design.
Proposed Method
In this section, we propose a novel single image denoising network, i.e. , MSANet, which simultaneously embraces the within-scale characteristics and the cross-scale complementarity of multi-scale features through the three neural blocks.
In the following, we will ﬁrst introduce the architecture principles of MSANet and then elaborate on the three neural blocks.
3.1 Architecture Principles
As illustrated in Fig. 2, MSANet employs an asymmetric encoder-subnetworks-decoder architecture.
Speciﬁcally, the encoder adopts four residual blocks [15] to extract features of four scales.
The ﬁrst residual block aims to extract the initial features without changing the resolution.
The other residual blocks respectively decrease the resolutions to half while increasing the channels to double.
With the multi-scale features extracted by the encoder, the subnetworks aim to exploit their within-scale characteristics via AFeB and AMB blocks.
As the characteristics of multi-scale features gradually change from high- to low-resolution, we take the two bottom and two top subnetworks in Fig.
2 as the high- and low-resolution branches, respectively.
The two bottom subnetworks need to handle the high-resolution features which usually are characterized by a mixture of details and noises.
It is highly expected to remove the noises without losing the ﬁne-grained image details, and thus AFeB is designed for adaptively preserving the indispensable details and ﬁltering unpleasant noises.
Meanwhile, another characteristic of high-resolution features is the limited contextual information.
The corresponding subnetworks should also enrich the contextual information while keeping the feature resolution unchanged in order not to lose details.
As a result, AMB is designed for adaptively extracting contextually informative features.
As AFeB and AMB could mutually boost each other, i.e. , low noise features from AFeB allow AMB to better capture the image contexts, while contextually informative features from AMB help AFeB to better distinguish the noises and details, we alternately stack AFeB and AMB to build the subnetworks.
In addition, as the higher-resolution features usually have fewer channels ( i.e. , fewer parameters), a deeper subnetwork is allowed to further alleviate their disadvantages, such as massive noises and limited contexts.
Different from the bottom subnetworks, the two top subnetworks aim to exploit the within-scale characteristics of the low-resolution features, which are characterized by less noise and more contextual information.
As the too-low resolution will destroy the image structure, the structure consistency cannot be guaranteed during the image recovery.
Hence, we employ AMB to capture more image contexts for enriching the contextual information without reducing the feature resolution.
In addition, as the low-resolution features are more robust to noise and have more channels, the top subnetworks mainly consist of AMB and are with shallower depth for efﬁciency.
Meanwhile, except for the lowest resolution, the ﬁrst and the last blocks in the top subnetworks are AFeB for adaptively controlling the input and output features.
To achieve the cross-scale complementarity, the decoder consists of four AFuB blocks followed by a convolution layer to output the recovered image.
More speciﬁcally, the ﬁrst three AFuB blocks increase the resolutions to double and decrease the channels to half, while adaptively sampling and transferring the ﬁne-grained detail features to the coarse-grained contextual features.
The last AFuB block keeps the resolution and channel unchanged, and directly samples and transfers the image details from the noisy input, which is an effective alternative to the global skip-connection, for which could avoid introducing noises from the noisy input.
For a given noisy input x , MSANet employs the encoder to extract the features of different scales and then feeds the features into different subnetworks to learn the scale-speciﬁc features.
After that, the decoder fuses the multi-scale features with varying characteristics from coarse to ﬁne to obtain the recovered image ˆ y = f
( x ) , where f ( · ) denotes MSANet.
To optimize MSANet, we employ the following objective function,
[Equation]
where y is the ground truth of x , and p = { 1 , 2 } .
3.2 Adaptive Neural Blocks
In this section, we elaborate on the proposed three neural blocks which are designed to learn better multi-scale features for single image denoising.
Adaptive Feature Block (AFeB).
To preserve the indispensable details and ﬁlter unpleasant noises, the block is expected to adaptively sample and weight the input features f i based on themselves, i.e. ,
[Equation]
where F ( · ) is used to compute the offset (∆ x, ∆ y )
w.r.t.
the positions ( x, y ) , as well as the corresponding weight ∆ w .
Then, the output features f i +1 could be aggregated by
[Equation]
where k is the number of samples, and w denotes the learnable weights.
In this way, AFeB could learn the sampling locations to indicate where are important for restoration, while assigning different weights to show how important the locations are.
As a result, AFeB preserves the indispensable details and ﬁlters unpleasant noises for better restoration.
However, it is prohibitive to traverse all positions for sampling and weighting w.r.t.
each position in the input features.
Hence, for the convenience of implementation, AFeB employs the modulated deformable convolution [55] to implement the aggregation operation in Eq.(3).
In detail, AFeB sets the sample number to the kernel size and the learnable weights as the convolutional weights.
Although this setting would decrease the number of samples and the ﬂexibility of weights, it is efﬁcient in the calculation of high-resolution features, and the limitations could be alleviated by stacking AFeB blocks.
In summary, AFeB consists of a convolution layer F ( · ) , a modulated deformable convolution, a LeakyReLU layer, a convolution layer and a skip-connection, i.e. ,
[Equation]
Adaptive Multi-scale Block (AMB).
Contextually informative features are highly expected for both the high- and low-resolution branches.
For the high-resolution features, reducing the resolution leads to the loss of image details.
For low-resolution features, a too-low resolution destroys the structure consistency of recovered images.
Therefore, to capture more contextual information without changing the resolution, we propose AMB by using several convolutions with different dilation rates.
Convolution with a large dilation rate could provide a large receptive ﬁeld, and multiple convolutions with different dilation rates could smoothly capture multi-scale information.
To reduce the cost caused by the multiple convolutions, AMB compresses the channels of each convolution so that the concatenated channels of all convolutions are equal to the output channels, i.e. ,
[Equation]
where f i and f i +1 are input and output features, and F dk is the k -th convolution with the dilation rate d .
As the concatenation assigns different scale features into different channels, the distinctive importance of multi-scale features is not considered.
To address this issue, AMB adaptively scales different channels and features, i.e. ,
[Equation]
where avg _ pool is an adaptive average pooling on space domain, mean denotes a mean operation along the channels, F fc is a linear layer, F conv is a convolution layer, F sig is a sigmoid layer, and 2 ∗ is used to control the ampliﬁcation ( > 1 ) or suppression ( < 1 ).
With the f i +3 , AMB passes it through a LeakyReLU layer, a convolution layer and a skip-connection, i.e. ,
[Equation]
Adaptive Fusion Block (AFuB).
As the high-resolution features contain a lot of disordered ﬁnegrained image details, and the low-resolution features contain abundant coarse-grained image contextual information, it is desirable to transfer the ﬁne-grained image details into the coarse-grained image context.
To this end, AFuB ﬁrst upsamples the coarse-grained features to the resolution of the ﬁne-grained features via
[Equation]
(8) where F TConv is a transpose convolution.
Then, to address the issue of the disordered details, AFuB adaptively samples and weights the ﬁne-grained features by using coarse-grained features to provide contextual information and ﬁne-grained features to provide details information, i.e. ,
[Equation]
where F ( · , · ) is used to compute the offset (∆ x, ∆ y )
w.r.t.
the positions ( x, y ) , as well as the corresponding weight ∆ w .
After that, AFuB transfers the ﬁne-grained details to the coarse-grained context via f finecoarse
[Equation]
where k is the number of ﬁne-grained detail features, w is the learnable weights.
Similar to AFeB,
AFuB employs a convolution layer as the function F and a modulated deformable convolution to achieve the aggregation in Eq.(10).
Finally, AFuB uses a convolution layer, a LeakyReLU layer, a convolution layer, and a skip-connection to further reﬁne features, i.e. ,
[Equation]
Experiments
In this section, we ﬁrst introduce the experimental settings, and then show the quantitative and qualitative results on nine datasets.
Finally, we perform analysis experiments including ablation study and feature-based visualization.
Due to space limitations, we present more experimental details and results in the supplementary material.
4.1 Experimental settings
We evaluate the MSANet on both real and synthetic noisy datasets.
For the evaluations on real noise, we employ the SIDD [1], RENOIR [3], Poly [42] datasets for training, and use SIDD Validation,
Nam [28] and DnD [30] datasets for testing.
For synthetic noise, we train MSANet on DIV2K [2] dataset, which contains 800 images of 2K resolution, by adding Additive White Gaussian Noise (AWGN) with the noise levels of 30, 50, and 70.
We use color McMaster [52] (CMcMaster), color
Kodak24 (CKodak24), color BSD68 [
We implement MSANet in Pytorch [29] and carry out all experiments on Ubuntu 20.04 with GeForce
RTX 3090 GPUs.
In our implementations, we use four scales of features with channels of 32, 64,and 256.
Moreover, we train MSANet 100 epochs via L 1 loss for real noise and 300 epochs via Lloss for synthetic noise.
Both real and synthetic noise training are with the batch size of 16 and the patch size of 128.
To optimize MSANet, the Adam [16] optimizer is used
[Table]
and the learning rate is initially set to 1e-4 and decays to zero via the cosine annealing strategy [23].
During the training, we randomly crop, ﬂip and rotate the patches for data augmentation.
In the testing, we employ PSNR and SSIM to evaluate the performance.
4.2 Comparisons on Real Noise Images
[Table] [Table]
[Table]
Real noise image denoising is challenging due to the real noise being usually signal-dependent and spatial-variant hinges on the in-camera pipeline.
Therefore, we carry out denoising experiments on three real noise image datasets, i.e. , SIDD Validation, Nam, and DnD. In brief, the validation dataset of SIDD contains 1,280 256 × 256 noisy-clean image pairs captured by the smartphone.
Nam includes 15 large image pairs with JPEG compression, and we evaluate MSANet on the selected512 × 512 patches by following CBDNet [14].
DnD contains 50 pairs of real noisy-clean images captured by cameras, and 1,000 512 × 512 patches are extracted for testing.
Due to the ground truths of the patches are not publicly available, we obtain the PSNR and SSIM results via the online submission system [30].
Besides, since JPEG compression makes the noise more stubborn on the
Nam, we ﬁrst train our model on the combination of SIDD and RENOIR for the evaluations on SIDD
Validation and DND, and then ﬁne-tune the trained model on the Poly for the evaluations on Nam.
We compare MSANet with 10 denoising methods that are comparable in model complexity, i.e. ,
CDnCNN-B, CBM3D [9], FFDNet+, CBDNet, N3Net [31], PD [54], PR [46], RIDNet [4], SADNet and DeamNet.
In experiments, we use the corresponding pretrained models provided by their authors, and refer to their results reported in the online submission system and papers.
Table 1 shows the quantitative results on SIDD validation dataset.
In brief, MSANet achieves the highest PSNR and SSIM values compared to other methods, e.g. , 0.85dB, 0.1dB, 0.09dB gains in PSNR, and 0.0066, 0.0015, 0.0013 gains in SSIM over the RIDNet, SADNet, and DeamNet, respectively.
For visual comparisons in Fig. 3, CBDNet and PD result in residual noises and pseudo artifacts, RIDNet, SADNet and DeamNet severely destroy the textures and obtain over-smoothed results.
In contrast, our method MSANet recovers textures and structures more subtly and obtains clearer restoration.
Some areas are highlighted by color rectangles and zooming-in is recommended for better visualization.
The quantitative results on Nam dataset are shown in Table 2, which demonstrates that our method achieves signiﬁcant improvements over the other methods.
Speciﬁcally, MSANet outperforms
RIDNet with 2.48dB (0.0049), SADNet with 0.6dB (0.0024), DeamNet with 1.49dB (0.0073) in
PSNR (SSIM) values.
For the visual comparisons shown in Fig.
4, our method obtains the best result for details recovery and noises removal, which is closer to the ground truth than other results.
Table 3 reports the quantitative results on DnD dataset, which are obtained from the DnD benchmark website.
From the table, one could observe that MSANet outperforms all methods both in PSNR and SSIM values.
Moreover, we further perform a qualitative comparison on the DnD dataset.
As shown in Fig.
the other methods achieve blurred results wherein many image details are corroded by noises
[Table]
while our method can effectively remove the noises and obtain clearer details
[Table]
4.3 Comparisons on Synthetic Noise Images
[Table]
We carry out experiments on three color and three grayscale noisy image datasets.
Speciﬁcally, the datasets are obtained by adding AWGN with the levels of 30,
[Table]
and 70 to the color and grayscale version of BSD68, Kodak24 and McMaster, respectively.
In brief, BSD68 contains 68 images which are frequently used for measuring image denoising performance, Kodak24 contains 24 images captured by ﬁlm cameras, and McMaster contains 18 images with statistics closer to natural images.
For comparisons, we choose seven representative denoising methods, i.e. , BM3D [10], DnCNN [50],
FFDNet [51], CLEARER [12], RNAN [53], SADNet [6] and DeamNet [32].
We call the python
[Table]
library for the evaluation of BM3D, and the pretrained models, provided by authors or retrained by us, for the evaluations of the other methods.
Table 4 reports the quantitative results on color image denoising, which shows that MSANet achieves the highest PSNR and SSIM values.
Taking the noise level of 70 as an example, our method can achieve PSNR gains about 0 .
03 ∼ 0 . 27 dB, and SSIM gains about 0 .
0010 ∼ 0 .
over the stateof-the-art methods, i.e. , SADNet, RNAN, and DeamNet.
Table 5 shows the quantitative results on grayscale image denoising.
From the table, one could observe that MSANet achieves the highest
PSNR values, and outperforms the other methods about 0 .
01 ∼ 3 . 59 dB w.r.t.
PSNR.
4.4 Analysis Experiments
The ablation study is conducted to demonstrate the signiﬁcance of utilizing the within-scale characteristics (WSC) and the cross-scale complementarity (CSC).
As shown in Table 6, “ED” and “ResB” are with homologous architectures, which use uniform Identity Mapping and Residual Block to process different scale features, respectively.
As using AFeB and AMB alone cannot exploit WSC well, “AFeB”/“AMB” and “AFeB+AFuB”/“AMB+AFuB” only slightly improve the performance over “ResB” and “AFuB”, respectively.
When using AFeB and AMB together, “AFeB+AMB” and MSANet ( i.e. , “AFeB+AMB+AFuB”) signiﬁcantly improve the performance over “ResB” and “AFuB”, verifying our claim on the role of AFeB+AMB w.r.t.
WSC.
Furthermore, thanks to CSC through AFuB, “AFuB” and MSANet ( i.e. , “AFeB+AMB+AFuB”) are signiﬁcantly better than “ResB” and “AFeB+AMB”, respectively.
In summary, the ablation study not only demonstrates the signiﬁcance of utilizing WSC and CSC, but also shows the effectiveness of our proposed solution.
[Table]
To further demonstrate the signiﬁcance of the within-scale characteristics, and the effectiveness of our solution in exploiting it, we show the qualitative comparisons on the intermediate multi-scale features before ( i.e. , without) and after ( i.e. , with) our subnetworks in MSANet.
From the Fig.
6, one could observe that our subnetworks could well exploit the within-scale characteristics, i.e. , preserving the indispensable details while ﬁltering the unpleasant noises for high-resolution features ( i.e. , 2nd and 3rd columns), and capturing rich contextual information for low-resolution features ( i.e. , 4th and 5th columns).
Besides, the different scale features show varying characteristics and signiﬁcant cross-scale complementarity, which would be further exploited by our AFuB.
Conclusions
In this paper, we proposed MSANet with three neural blocks, i.e. , AFeB, AMB, and AFuB, for single image denoising.
Different from existing multi-scale architectures, MSANet considers not only the cross-scale complementarity but also the within-scale characteristics, thus boosting the recovery performance as veriﬁed in experiments.
As this work could be regarded as ﬁnding the missing piece of multi-scale architecture design, we will explore other solutions to simultaneously exploit the within-scale characteristics and the cross-scale complementarity, and investigate their effectiveness in broader tasks such as deblur, segmentation.
Broader Impact Statement.
MSANet is a speciﬁcally designed architecture for supervised single image denoising, which requires intensive labor to collect a lot of noisy-clean image pairs, and thus has the potential to make more opportunities for employment.
However, MSANet is a general neural network and might be trained with uncertain data and used for uncertain purposes, such as watermark removal, which might prejudice the rights of others.
Besides, MSANet involves a novel idea of multi-scale architecture design and might be used to design new networks for uncertain purposes.
Moreover, the training and running of the model consume a lot of electricity causing carbon emission.
Acknowledgements
This work was supported in part by NSFC under Grant U21B2040, U19A2078, and 61836006; and in part by Sichuan Science and Technology Planning Project under Grant 2022YFQ0014.
Checklist
For all authors... (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?
[Yes] (b) Did you describe the limitations of your work?
[Yes] (c) Did you discuss any potential negative societal impacts of your work?
[Yes] See Broader Impact Statement.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes] 2.
If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
[N/A] (b) Did you include complete proofs of all theoretical results?
[N/A] 3.
If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes] See the URL of Github in Abstract.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[Yes] (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[No] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
[Yes] 4.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
[Yes] (b) Did you mention the license of the assets?
[Yes] It is detailed in the source code.
(c) Did you include any new assets either in the supplemental material or as a URL?
[No] (d) Did you discuss whether
and how consent was obtained from people whose data you’re using/curating?
[No] We use the public datasets in our experiments.
(e) Did you discuss whether the data you are using/curating contains personally identiﬁable information or offensive content?
[N/A]
We conduct experiments on the public datasets, which are widely used for research.
The discussion of personally identiﬁable information or offensive content could be referred to the data publisher.
If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
[N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
[N/A]