Exploring Data Efficiency in Image Restoration: A Gaussian Denoising Case Study

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-03-28 15:33:37

Abstract: Amidst the prevailing trend of escalating demands for data and computational resources, the efficiency of data utilization emerges as a critical lever for enhancing the performance of deep learning models, especially in the realm of image restoration tasks. This investigation delves into the intricacies of data efficiency in the context of image restoration, with Gaussian image denoising serving as a case study. We postulate a strong correlation between the model's performance and the content information encapsulated in the training images. This hypothesis is rigorously tested through experiments conducted on synthetically blurred datasets. Building on this premise, we delve into the data efficiency within training datasets and introduce an effective and stabilized method for quantifying content information, thereby enabling the ranking of training images based on their influence. Our in-depth analysis sheds light on the impact of various subset selection strategies, informed by this ranking, on model performance. Furthermore, we examine the transferability of these efficient subsets across disparate network architectures. The findings underscore the potential to achieve comparable, if not superior, performance with a fraction of the data—highlighting instances where training IRCNN and Restormer models with only 3.89% and 2.30% of the data resulted in a negligible drop and, in some cases, a slight improvement in PSNR. This investigation offers valuable insights and methodologies to address data efficiency challenges in Gaussian denoising. Similarly, our method yields comparable conclusions in other restoration tasks. We believe this will be beneficial for future research. Codes will be available at [URL].

Main paper: 
 related work 2.1 image restoration and gaussian denoising: 
 Image Restoration, a critical domain within low-level vision which tasked with unraveling the intricacies of elementary image features, such as edges, textures, and optical flow. Image Restoration tasks encompass a wide range of challenges, such as super-resolution [9, 21] , denoising [49] [50] [51] , deblurring [18, 41] , JPEG compression artifact removal [8, 10, 24] , low-light enhancement [19, 38] , and more. In general, the goal is to restore images degraded by specific factors to a higher quality. Gaussian image denoising, a fundamental challenge within the domain of image restoration, has been extensively studied to achieve the highest possible PSNR for images tarnished by noise. Besides, the learning behavior of models in image restoration tasks differs somewhat from high-level vision tasks. High-level vision tasks [25, 53] , such as image classification [6, 36] , typically involve models that perform stepwise feature extraction on images. These features are then processed through fully connected layers and softmax functions to output a vector representing the probabilities of belonging to different classes. In contrast, image restoration tasks, like Gaussian image denoising, often require models to establish relationships between current pixels and other pixels. Through these relationships and high-dimensional functions, pixel restoration is achieved. This training process cultivates the model's proficiency in identifying optimal restoration patterns. Despite the extensive exploration of image restoration techniques, a gap remains in understanding the impact of data on model learning. Specifically, there is a scarcity of research investigating data efficiency in image restoration, indicating a fertile ground for future inquiries.Reducing the size of the dataset while keeping as much information in the remained dataset has long been considered as a challenging problem and a considerable number of researchers have pushed progress in this area theoretically and practically. Data synthesis methods [4, 5, 29, 35, 52] (dataset distillation/condensation) have been the research hotspot recently. For example, Wang et al. [35] tried to synthesize a small dataset that can obtain a good performance in the testset by minimizing the classification loss. Another work by Zhao et al. [52] inspired by dataset distillation proposed to match the gradients and features to better learn how to synthesize images. Nevertheless, these approaches are computationally expensive and cannot provide a very satisfactory result. Another line of works focuses on data selection/pruning [28, 31, 33, 40, 43] , in which methodologies are usually more theoretically explainable. For example, Sorscher et al. [33] demonstrated breaking beyond power law scaling by developing a self-supervised pruning metric. Yang et al. [43] presented an optimization-based dataset pruning method based on influence function [17] and pruned 40% training examples on the CIFAR-10 dataset with only 1.3% test accuracy decrease.However, due to the distinct natures and learning patterns of classification tasks and image restoration tasks, it is imperative to rethink how to address data efficiency issues within the sphere of image restoration. 
rethinking the learning process in image restoration: 
 This section delves into the intrinsic learning behavior specific to image restoration tasks. We formulate the general pattern for a learning algorithm to solve a image restoration problem and define that a model's ability to generalize effectively is deeply rooted in the content information-like texture and edges-gleaned from training images. Using the Gaussian image denoising as a case study, we explore how content information influences the development of proficient denoising models uncovering both challenges and insights. Inspired by these observations, we venture into employing the influence function as a novel approach to address these issues.Notations. In the realm of image Blurring images reduces their clarity and informational content [18, 41] , posing challenges to human perception and computational models alike. This degradation effect allows us to investigate the role of content information in denoising model performance. Synthetic Dataset Construction. To study the impact of content information on model learning, we generate four synthetic datasets based on the BSD dataset [3] , each with a blur effect at a different level, from small to large. We use the mean filter (see Equation ( 3)) to blur images and set different kernel sizes to perform local convolutions on pixel regions, resulting in distinct levels of blurriness in the generated datasets (see Fig. 2 ).EQUATION𝑟 is the kernel size and 𝑠 = 𝑟 -1 2 presents the center shift. We use 𝑟 = {3, 5, 10, 20} to control blurriness in the generated data. We obtain four synthetic datasets D 𝑟 = {D 3 , D 5 , D 10 , D 20 }, and the blurriness increases with the value of kernel size 𝑟 . This aids in comprehending the connection between content information and the upper-bound performance of denoising models. Content Information Influences What Model Learns. We conduct experiments with Definition 1 on the synthetic data to validate Hypothesis 1. We train IRCNN [50] on the synthetic datasets and expect the model's performance on test datasets would inversely correlate with the kernel size 𝑟 , as less blurring datasets carry more distinctive content information than others. To evaluate this, we measure their performance over 1000k iterations training with a fixed learning rate 1e-4, calculating the PSNR/SSIM values on the Set12 [49] testset. Our findings, as detailed in Tab. 1, reveal a substantial decrease in PSNR/SSIM values, particularly for D 20 which exhibits the most significant blurring effect, resulting in a 4.81dB decrease.Remarkably, even the dataset with the least blurring, D 3 , exhibited a notable 0.83dB decrease in PSNR. These results underscore the critical role of distinct content information in guiding the learning process of denoising models. Consequently, this leads us to two However, due to 𝑛 is usually not small and the training denoising model needs a large number of iterations, therefore retraining the model for each removed 𝑧 is unacceptable and time-consuming.Fortunately, the research of influence function in machine learning theory [17] provides us with an accurate and fast estimation of parameter change caused by weighting an sample for training. For a training image 𝑧 and 𝑧 is weighted by a small 𝜂, the new parameters can be obtained through Equation ( 4 Thus assigning -1 𝑛 to 𝜂 is equal as removing the training image 𝑧 from original training dataset. A classic result [39] tells us that the influence of upweighting 𝑧 on the parameter θ can be calculated by Equation ( 5):EQUATIONwhere𝐻 θ 𝑑𝑒 𝑓 = 1 𝑛 𝑛 𝑖=1 ∇ 2 𝜃 L (𝑧 𝑖 , θ )denotes the Hessian and is positive definite by assumption. Then, we can linearly approximate the parameter change due to removing 𝑧 without retraining the model by computing θ-𝑧 -θ ≈ -1 𝑛 I up, param (𝑧) = 1 𝑛 𝐻 -1 θ ∇ 𝜃 L (𝑧, θ ). To further evaluate the influence of upweighting 𝑧 on the loss at a test point 𝑧 test , a closed-form expression can be obtained via the chain rule, indicated as Equation (6) .EQUATIONIn this way, we can efficiently approximate the loss change on the test dataset caused by removing 𝑧 as -1 𝑛×𝑚 𝑚 𝑗=1 I up, loss (𝑧, 𝑧 test 𝑗 ).Hessian-Vector Products. To calculate I up,loss (𝑧, 𝑧 test ), we need to invert -𝑠 test • ∇ 𝜃 L (𝑧, θ ). Because 𝑠 test is only related with 𝐻 -1 θ and gradient of 𝑧 test with respect to θ , so we can efficiently calculate I up,loss (𝑧 𝑖 , 𝑧 test ) across all training points 𝑧 𝑖 with only one calculation of 𝑠 test . Stabilized Stochastic Estimation. We use the stochastic estimation method developed by [1] to get an estimator that only samples a single point per iteration. Algorithm 1 provides the pseudo-code of stochastic estimation to calculate 𝑠 test . For each iteration, the HVP method uses two back-prop and one element-wise products to estimate hv (the product of Hessian and 𝑣), then 𝑠 estimate can be calculated by 𝑣 + 𝑠 estimate + ℎ𝑣/scale. In the initial use of the 𝐿 1 pixel loss (MAE, defined as || Î𝑖 -𝐼 𝑖 || 1 ) for gradient calculation, the problem arises due to the fact that the first-order derivative of the 𝐿 1 pixel loss is a constant value (if not zero point) that is not dependent on the current input values. Consequently, when this constant value is multiplied and summed with the HVP method and iteration operation, the estimated matrix values might become too large. This, in turn, leads to a numerical explosion during the computation of 𝑠 estimate . To stabilize the calculation, we originally propose to use the smooth function on the calculation of 𝐿 1 pixel loss as 1 2 || Î𝑖 -𝐼 𝑖 || 2 , if || Î𝑖 -𝐼 𝑖 || 1 < 1, as presented in Equation ( 7):𝐻 θ = 1 𝑛 𝑛 𝑖=1 ∇ 2 𝜃 L (𝑧 𝑖 , θ ),EQUATIONGiven the effective performance of the denoising model, we observe minimal absolute pixel discrepancies between the denoised image Î𝑖 and the original image 𝐼 𝑖 . This minimal deviation therefore contributes to the stability of the entire stochastic estimation process with our proposed stabilization method.  
experiment 4.1 experiment setup: 
 We perform experiments on grayscale Gaussian image denoising with thorough analysis (Similar conclusions on other restoration tasks can be found in supplementary materials). We assess the data efficiency across four benchmark training datasets: BSD500 [3] (400 images), WED [26] (4744 images), Flickr2K (2650 images) [34] , and DIV2K (900 images) [2] , referenced respectively. We employ two CNN architectures, IRCNN [50] and DnCNN [49] , alongside the powerful vision transformer model Restormer [44] , with Set12 [49] and BSD68 [27] testset, to evaluate their performance in our experiments. All experiments are conducted in PyTorch framework with For IRCNN, the Mid subset sourced from WED consistently delivers the highest PSNR across the evaluated subsets, achieving superior performance at noise levels 15 and 50 with only a minimal performance decrement at noise level 25, despite utilizing merely 3.89% of the dataset. This efficiency in data usage while maintaining or enhancing performance metrics underscores the effectiveness of our subset selection methodology.Cross-model training further validates our approach; the Mid subsets consistently outperform training on the full dataset for the DnCNN model. In the case of Restormer, despite the dataset comprising only 2.30% of the original, the decrease in PSNR is marginal: minimal to 0.03 dB for noise level 50. These results indicate the transformer model's heightened sensitivity to image size, with larger images from Flickr2K and DIV2K contributing to its performance due to increased pixel availability. Conversely, CNN architectures, such as DnCNN, show a preference for efficient subsets from WED, suggesting a nuanced interaction between data efficiency and network architecture. This exploration indicates that the principle of optimal data efficiency is also intricately linked to the architecture of the network.Building on the observations detailed in Tab. 3, where IRCNN and DnCNN models trained on Mid subsets surpassed the performance of models trained with full datasets, we note a slight performance decrement in Restormer under similar conditions. This subsection delves deeper into the Restormer's adaptability and performance nuances. Evaluating The Impact of Mid Subset Size Variations. A key area of inquiry pertains to the potential of achieving or exceeding full dataset performance through adjustments in the size of the Mid subset. This exploration is motivated by the desire to understand the implications of such modifications on model performance. Subsequently, we conducted experiments with Mid subset sizes of K In the preceding section, we have presented the primary experimental findings and highlighted the generalization of Mid subsets across various network architectures. In this section, for a better understanding of model's learning, we present thorough analysis experiments on both Top, Bottom and Mid selection methods. First, we conduct experiments under more constrained data settings, we shrink dataset to a smaller size and compare their performances on Top, Mid, Bottom subsets. Then, we comprehensively analyze model's learning behavior with these three subsets, finding that Mid subset presents both faster convergence speed and better convergence result. Furthermore, we offer visualization of frequency spectrum from three subsets and provide a perspective from frequency and gradient to understand how and why Mid selection outperforms other selection methods. All analysis and subset selections are conducted on original WED [26] with noise level 50 unless mentioned otherwise.What If We Further Shrink Three Training Subsets? It's instructive to explore the difference on model performance if we reduce the selection size of three subsets from K = 200 to a lower number. In this case, we train IRCNN with noise level 50 on less data and record their corresponding best PSNR performances. As shown in Tab. 5, we can discover two phenomenons:• Mid selection method obtains the best PSNR performance in all experiments. • The smaller the dataset size is, the larger the Max Diff among the three selection methods becomes, this demonstrates that data efficiency possesses a much a higher status in scenarios lack of training data. basis data provides coarse-grained information about the target function, while outliers provide fine-grained information. It turns out that neither keeping the hardest samples nor keeping the easiest samples is the best way to obtain the most efficient data in Gaussian image denoising task. Contrary to what might be intuitive, the most effective strategy involves preserving the samples that exhibit smooth characteristics (referred to as the Mid subset).Relation to Frequency and Gradient. In order to gain a deeper understanding of the underlying principles, we select examples from three subsets and apply the Fast Fourier Transform to obtain their corresponding spectrum. As shown in Fig. 6 , we can see that high frequency information visually differs in (a)/(b)/(c) which represent examples from Bottom/Mid/Top subsets respectively. Top subset example contains more high frequency information, while Gaussian noise is also high frequency information, thus increase the difficulty for model learning. To obtain a more statistical results, we average all examples with three criteria: 1. gradient based image quality assessment method (IQA [47] ) 2. high frequency average pixel intensity (HFAPI, with spectrum center masked). 3. low frequency average pixel intensity (LFAPI, with spectrum center reserved). Tab. 6 presents the quantitative results. It is observed that the Mid subset exhibits IQA value that is relatively close to the Bottom subset. Simultaneously, the Mid subset contains highfrequency information of almost the same intensity as the Top subset. However, the intensity of low-frequency information in the Mid subset is comparatively lower. Conversely, the Top subset possesses the strongest high-frequency information, but its overall IQA is significantly lower than that of the other two subsets. This observation underscores a limitation in the model's performance on the Top subset. Moreover, the Bottom subset showcases the highest IQA value. This implies that the model is more adept at learning the coarse-grained information provided by the Bottom subset, however, it's difference between the intensity of high-frequency and low-frequency information is minimal among three subsets, which subsequently leads to a lower final result in comparison to the Mid subset.  
conclusion: 
 In this paper, we navigate the intricacies of data efficiency in the realm of image restoration, with a spotlight on Gaussian image denoising as a pivotal case study. Our comprehensive analysis reveals the profound influence of content information encapsulated within training datasets on the performance of image denoising models. By introducing a novel, stabilized methodology for quantifying this content information, we have enabled a strategic ranking of training images based on their score intervals. Additionally, we perform an in-depth analysis to understand the model's learning behavior under diverse situations. Our experimental results demonstrate that models can achieve, or even exceed, the performance of fully trained counterparts with a fraction of the training data, challenging traditional views on data requirements. We hope our methodological framework and insights will serve as valuable resources for future research.  
: 
 testset. We trained representative CNN/transformer models on both the original datasets and efficient subsets with same iterations for fair comparison. Our results underscore the potential to match or surpass fully trained models with significantly fewer training images.Recent trends reveal a surge in model complexity [11, 12, 32, 46] , particularly with the advent of large-scale vision models [13, 14, 16, 42] , necessitating substantial data and computational resources. Although data has consistently remained a pivotal factor influencing deep learning models, the approach adopted by researchers in the domain of low-level vision predominantly falls into one of two categories: either they modify model architectures based on publicly available datasets to achieve state-of-the-art results on popular tasks [22, 37, 44, 48] , or they curate bespoke datasets and innovative methodologies for less mainstream tasks [20, 23, 30] . Notwithstanding these developments, there remains a conspicuous gap in the exploration of data itself, particularly in terms of efficiency in the context of image restoration tasks. While some studies like [49, 50] have marginally touched upon the impact of enlarging training data, suggesting only modest enhancements in model performance on certain testsets, a comprehensive exploration into how data influences model learning is markedly absent. This gap underscores the potential for significant advancements in understanding and exploiting data efficiency in this field.In this paper, we investigates data efficiency in image restoration tasks, using Gaussian image denoising as a case study. Through fundamental hypothesis-driven experiments, we initially substantiate that content information within the images is pivotal for acquiring a perfect denoising model. Our investigation reveals variations in content information complexity across dataset images, highlighting the importance of selecting the most efficient images for training.Notably, there is currently no concrete method for quantitatively analyzing content information numerically, and removing samples one by one would necessitate 2 𝑛 model retraining, rendering it impractical. To address these challenges, we employ statistical machine learning techniques to introduce a content information quantification method based on influence function [17] . Specifically, influence function enables the estimation of the impact of training data on test results without the need for retraining the model. We leverage the Hessian-vector product (HVP) method in influence calculation, and propose a stabilized computation process that is particularly suited to image restoration tasks. Through comprehensive experiments, we elucidate the principles of efficient subset selection based on influence scores, assess their applicability across diverse network architectures, and provide an in-depth analysis of model learning behaviors on these subsets. Our findings demonstrate the feasibility of achieving, and in some instances surpassing, the performance of fully trained models with a fraction of the data (see Fig. 1 ). In summary, the main contributions of this paper are as follows:• We provide a comprehensive exploration of data efficiency for image restoration research using Gaussian image denoising as a case study, we investigate the effects of content information on model performance, point out that their variations exist inside training datasets and introduce an effective, stabilized method to estimate the influence of each training image. • We offer an insightful analysis on efficient subset selection based on influence scores, validating their generalization across different network structures, and conducting extensive studies on models' learning behaviors. Our findings reveal that subsets with mid-level influence scores consistently yield optimal test performance. • Our experimental outcomes underscore the potential to match or exceed the performance of comprehensively trained IR-CNN and Restormer denoising models with merely 3.89% and 2.30% of the original dataset, respectively. To our knowledge, this represents the first in-depth exploration and insightful analysis of data efficiency in the domain of image restoration. 
 
 Appendix:  


Decision: The authors provided a response to the reviewers' expressed concerns.

Reviewer Mzmz finds it unsatisfactory as he still has concerns with regard to clarity and experimental validation.

The other three reviewers recommend acceptance, weighing in the ideas, the solution, and the quantitative results.

After carefully checking the paper, the reviews, the rebuttal, and the post-rebuttal justifications, the Meta-Reviewer agrees with the reviewers that paper is "moderately original" (Reviewer Mzmz), presents "comprehensive experiments" and "in-depth analysis" (Reviewer kt8g,Reviewer UE8S), "it is easily understandable" (Reviewer SNZ9), and has practical value (Reviewer kt8g,Reviewer UE8S).

The strengths outweigh the weaknesses and the Meta-Reviewer agrees with the majority of the reviewers that this work makes significant contributions of interest to the community and invites the authors to further improve their work and integrate part of the contents from their response and address the remaining issues in the camera ready paper.

Reviews: review: paper_summary: paper_summary: The paper explores data efficiency in image restoration, focusing on Gaussian image denoising. It emphasizes the significant impact of content information, such as edges and textures, on model performance. By ranking training images based on their influence scores, the paper identifies efficient subsets for training. Experimental results demonstrate that models can achieve comparable or superior performance using a fraction of the original data. The study provides valuable insights and methods for addressing data efficiency challenges in image restoration tasks.
strength_weakness: strength_weakness: 1. This paper introduces a novel and stabilized method to quantify content information in training images, enabling the ranking of images based on their influence.

2. The paper presents comprehensive experiments to validate the hypothesis, explore efficient subset selection strategies, and assess the transferability of efficient subsets across different network architectures.

3. The paper provides an in-depth analysis of model learning behaviors on efficient subsets, shedding light on why mid-level influence subsets consistently yield optimal test performance.

4. The proposed method shows generalization across different network architectures and datasets, indicating its potential applicability to a wide range of image restoration tasks.
limitations: limitations: Some points need to be clarified:

1. How to define the "empirical risk function" in  Equation (1)?

2. Since exploring the effects of content by blurring the ground truth, what is the proposed method's effect in the debluring task?

3. The data selection methods, as a data pre-processing manner, should discuss the calculation load for a fair comparison with those network design manners.

4. Some mistakes in Figure 3 texts.
paper_summary: paper_summary: The paper provides a thorough investigation into data efficiency in image restoration research, using Gaussian image denoising as a case study. This paper explore the impact of content information on model performance and introduce a stabilized method for estimating the influence of each training image. This paper offers insightful analysis on efficient subset selection based on influence scores, demonstrating their generalization across different network structures and conducting extensive studies on models' learning behaviors. The experimental results highlight the potential to achieve or surpass the performance of comprehensively trained models with significantly reduced datasets.
strength_weakness: strength_weakness: 1. This paper offers a comprehensive examination of data efficiency in image restoration, providing valuable guidance for further optimizing training data and strategies.
2. The analysis of learning behaviors from a data perspective contributes to a deeper understanding of neural network learning mechanisms.
3. The effective and stabilized method for quantifying content information appears to be highly practical.

Benifiting from my extensive industrial experience, I understand and endorse the majority of the discoveries presented in this paper. The discoveries described align with findings from numerous industrial practitioners over the years, which deserve broader recognition.
limitations: limitations: As stated above, I can promise that the majority of the discoveries presented in this paper have been confirmed to be true in the industrial field. Unfortunately, I have also found many issues with this paper, especially when I consider it as an academic paper.

1. In L105, the paper claims that *"a comprehensive exploration into how data influences model learning is markedly absent,"* which is actually incorrect. A series of papers closely related to raw image processing, such as those on noise modeling, have long focused on *"how data influences model learning."* I do not find any keywords related to *"raw"* throughout the paper, indicating that this paper significantly lack of research in this area.
I recommend two directly relevant raw denoising works here:   
    a) SFRN (*Rethinking Noise Synthesis and Modeling in Raw Denoising*), which provides superior experiments in Sec. 3.2.   
    b) PMN (*Learnability Enhancement for Low-Light Raw Image Denoising: A Data Perspective*), its diversity analysis of datasets and data mapping learnability analysis are highly relevant to this paper.   
Many conclusions presented in this paper can be pieced together from various papers in the field of degration modeling.

2. The paper focuses on analysis, and the proposed methods are merely minor improvements based on influence estimation theory, lacking novelty from an academic perspective. Furthermore, selecting mid-influence images as a subset is an overly tricky approach, making the criterion unstable. For instance, if I extract low-influence images from a massive dataset as a new dataset, the meaning of mid influence in the new dataset would obviously differ from that in the original dataset.

3. The paper's title risks overclaiming, as conclusions drawn from image denoising may not necessarily generalize to image restoration. Image denoising heavily relies on noise priors, which may differ from tasks relying more on image priors (such as super-resolution). At least, the compressibility of the dataset should be heavily discounted.

4. The writing style of the paper adds to the difficulty of reading, which I would describe as "loan-style introduction." I often encounter confusing descriptions with explanations that are far from the description (position), greatly disrupting the reading experience.

### Additional Comment

The paper raises a very good question, and I love it. It initially impressed me, and I expected to judge it as "Weak Accept" at first. However, unfortunately, it did not provide much incremental information for me, and there are many problems. I believe this question should be addressed in a better way, and I look forward to seeing a revised version of the paper in the future.
paper_summary: paper_summary: This investigation introduces an effective and stabilized method for quantifying content information and delves into the intricacies of data efficiency in the context of image restoration.
strength_weakness: strength_weakness: The paper is sound, clear motivation, well-organized, and articulately presented, ensuring it is easily understandable.
limitations: limitations: 1. Data size is key to model performance, especially in the era of large models. In the paper, while training based on multiple datasets and verifying the impact of data effectiveness. (BSD500 (400 images), WED (4744 images), Flickr2K (2650 images) , and DIV2K (900 images)). However, this level of data size is clearly insufficient compared to the data size of today's large models. This is something the author might consider elaborating on. 


2. Minor issue: inconsistent formatting of references:
In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops.
In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
In Computer Vision – ECCV 2020, Andrea Vedaldi, Horst Bischof, Thomas Brox, and Jan-Michael Frahm (Eds.)

3. Some arXiv references can be replaced with published versions.
paper_summary: paper_summary: This paper focuses on exploring data efficiency issues specific to low-level image restoration. It uses Gaussian image denoising as a case study in an attempt to dramatically compress data size while maintaining model performance. Specifically, the authors first investigate the effect of the content information contained in the training dataset on the performance of typical image denoising models. The authors then quantify this content information by strategically ranking the training images based on score intervals. In the end, the experimental results demonstrate the feasibility of the proposed method.
strength_weakness: strength_weakness: 1. Novelty. Moderate original.
2. Technical Correctness. Probably correct.
3. Reference to Prior Work. References adequate.
limitations: limitations: 1. Clarity.

a) The authors have used some overly ornate expressions in this manuscript that prevent the reader from understanding the content of the current paper clearly and easily. 

b) The authors should carefully explain the differences between the proposed methods and previous related studies, such as dataset distillation (or dataset condensation), coreset selection, and so on, instead of giving a general conclusion that the performance of these solutions is unsatisfactory.

2. Experimental Validation.

a) As mentioned above, a comparison of different dataset compression methods is required to demonstrate the superiority of the proposed subset selection strategy. This is possible even if the relevant studies are performed in ablation experimental sessions.

b) Considering that the research objective is the image restoration task, some visual comparisons are thus needed to validate the effectiveness of the proposed method, i.e., the ability to maintain a comparable performance to the original model by only training on the selected images. However, these are missing in the current manuscript.