BCSCN:Reducing Domain Gap through Bézier Curve basis-based Sparse Coding Network for Single-Image Super-Resolution

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-02 08:24:01

Abstract: Single Image Super-Resolution (SISR) is a pivotal challenge in computer vision, aiming to restore high-resolution (HR) images from their low-resolution (LR) counterparts. The presence of diverse degradation kernels creates a significant domain gap, limiting the effective generalization of models in real-world scenarios. This study introduces the Bézier Curve basis-based Sparse Coding Network (BCSCN), a preprocessing network designed to mitigate input distribution discrepancies between the training and testing phases of super-resolution networks. BCSCN achieves this by removing visual defects associated with the degradation kernel in LR images, such as artifacts, residual structures, and noise. Additionally, we propose a set of rewards to guide the search for basis coefficients in BCSCN, enhancing the preservation of main content while eliminating information related to degradation. The experimental results highlight the importance of BCSCN, showcasing its capacity to effectively reduce domain gaps and enhance the generalization of super-resolution networks.

Main paper: 
 introduction: 
 Single Image Super-Resolution (SISR) is a fundamental problem in computer vision, with the goal of reconstructing high-resolution (HR) images from their corresponding low-resolution (LR) counterparts. In response to the surge in deep learning, approaches leveraging deep neural networks have garnered significant attention for addressing image super-resolution (SR) tasks. These methods typically rely on collected HR-LR image pairs for training [3, 12, 15, 17, 23, 24, 34, 43] . However, obtaining such pairs in realworld scenarios can be arduous, often requiring the generation of LR images from HR originals. Most SR techniques resort to simplistic bicubic downsampling to create synthetic HR-LR pairs, but this methodology introduces a significant challenge: the domain gap emerges due to the disparity in the distribution of LR images between the training and testing phases, stemming from the use of distinct degradation kernels [21, 32, 37] . This domain gap hinders the models' capacity to generalize effectively, impacting their practical utility in real-world applications where the degradation kernels are intricate and unknown, differing from those applied in LR image generation during training.In most cases, three distinct visual defects emerge during the degradation process of an image: Artifacts: These are isolated artifacts generated during image compression or processing that are unrelated to the image content. Residual structures: These are isolated structures that form due to the severe loss of semantic contextual information during the image degradation process. Noise: This includes common Gaussian noise models and non-Gaussian noise caused by camera sensors. However, for different degradation kernels, such as the real degradation processused in LR image generation, these visual defects may manifest differently. This variation is the root cause of the domain gap.This raises the question of whether there exists an image preprocessing operation, independent of the degradation kernel, capable of eliminating visual effects associated with the degradation kernel. These effects include artifacts, residual structures, and noise, while concurrently preserving content to alleviate the domain gap. The objective is to attain a "clean" image through this operation, one that is imperceptible in terms of degradation, yet retains abundant content and detail. Subsequent to this preprocessing step, a SR network with superior generalization performance can be trained using these "clean" images, effectively learning the mapping from "clean" images to HR images.In this paper, to achieve this goal, we present a preprocessing neural network, the Bézier Curve basis-based Sparse Coding Network (BCSCN), designed to generate the "clean" image. BCSCN integrates an over-complete Bézier curve basis space, representing degraded LR images through linear combinations of sparse Bézier curve basis functions. To determine the basis coefficients for a given LR image, BCSCN employs a basis coefficients search agent. The ultimate "clean" image with sharp edges is reconstructed based on the LR image's bases and coefficients using a differentiable neural renderer within BCSCN.The key innovation lies in utilizing a well-designed and restricted/sparse set of Bezier curve basis functions. This choice allows for the exclusion of details associated with the degradation kernel, such as artifacts, residual structures, and noise. Consequently, the final reconstruction not only preserves the essential image content but also effectively eliminates information related to the degradation, as illustrated in Figure 1 .Importantly, BCSCN is independent of the degradation kernel. It can be trained on samples generated by a single, simple degradation kernel (such as bicubic degradation kernels) and applied to LR images generated by other degradation kernels to obtain their corresponding "clean" images.In summary, the primary contributions of this article can be outlined as follows:  
related work 2.1 single image super-resolution: 
 SRCNN [5] introduced a three-layer neural network and standed as a pioneer in SISR. Since then, researchers turned to focus on developing larger and deeper networks using residual architectures. For example, a 20-layer deep neural network was used by VDSR [12] based on residual learning. EDSR [17] took a step further, featuring a remarkably deep and wide network with modified residual blocks for SR tasks. RCAN [43] introduced channel attention and further designed a network with 1000 layers. However, convolutional sliding window mechanism limits the ability of a network to harness global contextual information. Hence, HAN [24] incorporated layer-wise, channel-wise, and spatial attention modules, capturing hierarchical features by considering inter-layer correlations. Recently, new methods have been developed to capture long-range relationships between pixels in images. This allows them to have larger receptive fields and achieve better PSNR performance [3, 15, 23] . Nevertheless, these methods exhibit sensitivity to distribution discrepancies in LR images, resulting in artifacts when inference distribution deviates from the training distribution, thereby severely constraining their real-world applicability.In an effort to narrow the domain gap between training and real data samples, some researchers have explored the degradation process of images to align training data distribution with real-world test data distribution. Some have achieved this by adjusting camera focal lengths, directly capturing paired data under real-world conditions [1, 36] . Other researchers have used methods that rely on predefined degradation kernels. These methods synthesize degraded images by artificially creating complex degradation kernels that aim to cover the distribution of datasets from various real-world scenarios [18, 21, 31, 33, 40, 41] . To generate LR images that closely match specific real test image domains, some approaches [22, 32, 37] have opted for adaptive learning of degradation, leveraging GANs to explicitly or implicitly learn the degradation process from real target distributions during training, thus bringing training samples closer to specific target real data distributions. we leverage the robustness of sparse coding against the impact of degradation factors, effectively addressing distribution discrepancies between synthetic and real data, all without the need for prior knowledge of the specific degradation distribution. Our approach, trained solely on data synthesized using simple bicubic downsampling, significantly enhances model generalization performance.Sparse coding has been extensively explored as a conventional approach in the field of computer vision. Previous methods [6, 14, 38, 39] typically involved learning an overcomplete dictionary, from a set of training images. During the reconstruction phase, the optimization process focuses on finding a set of coefficients that correspond to the atoms of the dictionary. These coefficients aim to yield the best linear combination to reconstruct the test images. However, due to the challenge of learning a sufficiently overcomplete dictionary, its reconstructive capabilities are limited when applied to natural images. In this paper, we define a sparse dictionary as a continuous basis space composed of second-order Bézier curves. We employ a reinforcement learning agent to explore sparse basis coefficients, enabling accurate reconstruction of LR images while alleviating degradation factors such as residual structures, noise, and artifacts. 
method: 
 By introducing our proposed BCSCN, the super-resolution (SR) process is divided into two stages: the initial BCSCN preprocessing stage and the subsequent SR network stage, as illustrated in Figure 2(a) . The BCSCN incrementally reconstructs the LR image 𝐼 into a "clean" image Î𝑇 by searching for the optimal basis coefficients within a predefined basis space. Subsequently, the resultant "clean" image Î𝑇 is fed into the SR network to produce the output HR image.Given over-complete bases {𝜙 𝑖 (•)|𝑖 = 1, • • • , 𝑁 }, a degraded LR image 𝐼 can be represented by:EQUATIONwhere 𝛼 is the coefficient of the basis functions. In this paper, we opt for a space of Bézier curve bases with diverse attributes, including shape, width, and color, as the foundations. And a basis of this space is defined as:IEQUATIONwhere B 𝑖 denotes the shape, defining as a second-order Bézier curve that is controlled by a set of specified control points 𝑃 𝑖 0 , 𝑃 𝑖 1 , 𝑃 𝑖 2 :EQUATIONBy flexibly adjusting the positions of these control points, various natural image elements such as points, lines, and curves with sharp edges can be formed.And in Eq. 2, 𝑤 𝑖 signifies the curve's width, and 𝐶 𝑖 is a threeelement tuple representing its color attributes (𝑟, 𝑔, 𝑏). This enhancement amplifies the expressive capacity of the Bézier curve, allowing for a better fit to natural shapes.As discussed in Section 1, our approach involves employing a carefully designed and sparse set of bases, which proves effective in preserving edge details while eliminating defects introduced by degradations. The utilization of sparse codes for image representation, with a significant portion of coefficients 𝛼 𝑖 being zero, allows the model to focus on crucial features in the input image, thereby effectively removing defects associated with degradation.To extract the main content from the image, a small number of bases are selected from the entire set to reconstruct a LR image. Sparsity control operates on two aspects: the area of the curves and the number of curve bases. By managing these aspects, the decomposed bases can effectively capture the main content of the image.In this paper, we enforce constraints on the basis coefficients to be either one or zero, preventing the combination of multiple bases from forming a transparent element -an unusual occurrence in real images. This transformation shifts the decomposition process from calculating basis coefficients to a basis selection procedure, significantly reducing the complexity of our proposed method. However, it is important to note that this remains an NP-hard problem, and we address this challenge by utilizing reinforcement learning methods.Furthermore, we model the base selection procedure as a sequential decision-making procedure, utilizing a base coefficient search agent to minimize the reconstruction error between the "clean" image and the LR image. The objective is to identify the optimal Bézier curve base representation within a fixed number of bases. Learning-ToPaint [10] has already confirmed the effectiveness of this method in efficiently reconstructing realistic and natural images in complex environments.As depicted in Figure 2 (b), given the LR image 𝐼 and the current reconstruction result Î𝑡 , the agent's objective is to predict the parameters of the next basis 𝑎 𝑡 based on Î𝑡 and 𝐼 .Next, we will delineate the state space and action space for the basis coefficient search agent, as well as the reward function.The state space is composed of three components: the LR image 𝐼 , the current reconstruction image Î𝑡 , and the current step 𝑡. It is represented as a triplet: 𝑠 𝑡 = {𝐼, Î𝑡 , 𝑡 }.Bézier curve bases are determined by a set of parameters, and the action space of the agent consists of these parameters, as depicted below:EQUATIONwhere {𝑃 𝑡 0 , 𝑃 𝑡 1 , 𝑃 𝑡 2 } represent the control points of the shape of 𝜙 𝑡 , 𝑤 𝑡 is the width of the second-order Bézier curve, and (𝑟 𝑡 , 𝑔 𝑡 , 𝑏 𝑡 ) signifies the color attributes of the basis. We employ a pre-trained differentiable neural renderer [10] , which takes the bases parameters 𝑎 𝑡 as input and produces the pixel-space Bézier curve basis 𝜙 𝑡 as output. Subsequently, 𝜙 𝑡 is superimposed onto 𝐼 𝑡 to yield the next reconstructed image 𝐼 𝑡 +1 , resulting in the next state 𝑠 𝑡 +1 = {𝐼, Î𝑡+1 , 𝑡 + 1}. EQUATIONwhere 𝑅(𝑠 𝑡 , 𝑎 𝑡 ) represents the reward function at step 𝑡, and 𝑟 (𝑠 𝑡 ) is defined as the reward measure of the operation results Î𝑡 at step 𝑡 in relation to 𝐼 .Inspired by [10] , we employed the WGAN with gradient penalty (WGAN-GP) [7] as the reconstruction distance measure between the current reconstruction image Î𝑡 and the target image 𝐼 , specifically defined as:EQUATIONwhere 𝐷 (•) is the discriminator.Additionally, to encourage the agent to choose a basis which represents a larger curve area, ensuring sparsity, capturing prominent image features, and effectively mitigating degradation factors in image reconstruction, we introduced a curve area reward:EQUATIONwhere S(•) is used to compute the area occupied by the curve represented by basis 𝜙 𝑡 in the reconstructed image. However, the experimental findings reveal that when 𝐼 is exclusively reconstructed using bases that represent large curves, it can lead to over-smoothing. This results in the preservation of only the main structures while fine details are sacrificed. Therefore, we further introduced high-frequency response reward to encourage agents to use bases with smaller curve in the areas with rich details, to preserve more details in the reconstruction results. This reward can be expressed as:EQUATIONwhere Laplace(•) denotes the Laplace operator used to extract the second-order gradient of the image, enhancing the response to fine image details. Ultimately, the reward measure 𝑟 (𝑠 𝑡 , 𝑎 𝑡 ) is defined as the weighted sum of all rewards:EQUATIONwhere, the weights 𝜆 1 , 𝜆 2 , and 𝜆 3 are chosen empirically to maintain a balanced scale across the three rewards.The architectural configuration of the BCSCN is illustrated in Figure 2 , employing an Actor-Critic framework comprising the Actor network, Critic network, and neural renderer. The specific structure is detailed as follows: Both the Actor and Critic networks adopt a residual structure similar to ResNet-18 [8] . The Actor network is responsible for generating basis function parameters based on the current state 𝑠 𝑡 , while the Critic network provides an evaluation score based on the current state 𝑠 𝑡 , the current reconstructed image Î𝑡 , and the next-step reconstructed image Î𝑡+1 . The differentiable neural renderer employs a 4-layer fully connected network and two layers of convolutional networks to map the parameters of the basis functions to rasterized curve basis, which are then overlaid with Î𝑡 , to generate the next-step reconstructed image Î𝑡+1 . During the training of the BCSCN, the Deep Deterministic Policy Gradient (DDPG) [16] training strategy is employed. The parameters of the differentiable neural renderer are fixed, while the remaining components undergo joint training.In addition, due to the rich structural information in natural images, accurately reconstructing images at arbitrary scales poses a significant challenge. Leveraging non-local priors inherent in natural image, local patches from different positions exhibit repetitive patterns. Similar to most SR methods which divide the entire image to small patches for training, in our BCSCN training and inference processes, we decompose the input image into consecutive small patches for individual reconstruction, thereby mitigating the difficulty of basis search and achieving more precise reconstruction. 
experiments: 
 To assess the impact of our proposed approach on improving the generalization of SR networks, we integrated BCSCN with SR network, resulting in a new SR network called BCSCN-SR. The training of BCSCN-SR was conducted in multiple stages. Initially, BCSCN underwent training using the reward function specified in Eq 9. Once this stage was completed, the subsequent SR network could be trained based on the preprocessed data.Three experiments were devised to assess the effectiveness of the proposed method: 1) assessing the impact of BCSCN on domain gap narrowing, and 2) validating the generalization of BCSCN-SR, and 3) to compare it with blind super-resolution approaches.Training Set: We utilized the DIV2K dataset [28] to train BCSCN, which comprises 900 HR images. Among these, 800 images were allocated for training, and 100 for validation. The training set was composed of LR images generated by performing bicubic downsampling on the 800 HR images from the DIV2K dataset.Testing Set:We employed four Real World Super-Resolution Challenge (RWSRC) datasets provided by NTIRE and AIM to assess our method. These datasets, namely Ntire2018 Track2 [30] , Ntire2018 Track4 [30] , Aim2019 Track2 [20] , and Ntire2020 Track1 [19] , were obtained by downsampling (×4) the HR images from the DIV2K validation set using four distinct methods. The degradation operators used for downsampling are diverse and undisclosed to simulate real-world degradation scenarios, as detailed in the respective papers. Additionally, we incorporated the RealSRSet dataset by Kai et al. [41] , featuring 20 real LR images without HR counterparts.To train BCSCN, We randomly cropped patches of size 16×16 from LR samples. The training batch size was set to 96. The agent's replay buffer size was configured as 16,000, allowing a maximum prediction of 20 time steps. The learning rate was initialized to 1 × 10 -4 . After BCSCN training, we utilized it to preprocess LR images, producing "clean" images for subsequent SR training.For the BCSCN preprocessing network, we utilized SIFID [27] to assess its capability in narrowing the distribution discrepancies of degraded images. SIFID has been proven sensitive to variations in The term "clean" denotes images processed through BCSCN.degradation kernels [25] . To evaluate the performance of BCSCN-SR, we employed PSNR and SSIM [35] evaluation metrics on the Y-channel. Additionally, we incorporated perceptual metrics, including LPIPS [42] based on the AlexNet [13] to gauge the perceptual similarity on the RGB channels, and FID [9] was employed to quantify the distance of distributions between the original and the reconstructed images. For RealSRSet, where the ground truth is not available, we employed no-reference image quality measures oriented towards perceptual evaluation, specifically NIQE [42] and MUSIQ [11] for evaluation. Human perception plays a pivotal role in assessing the effectiveness of super-resolution techniques. In line with previous studies [2, 32, 33] , we prioritize perceptual metrics, regarding PSNR and SSIM metrics solely as references.To assess BCSCN's capability in narrowing domain gaps, we measured the disparities in LR image distributions between the training and testing sets, both with and without the application of BCSCN. Specifically, we quantified the dissimilarity in the LR image distribution between the training set and each testing set. Additional, we examined distribution disparities between "clean" images from the training set and each test set. These images undergo preprocessing by BCSCN before being input into SR network of BCSCN-SR. Table 1 presents the differences in SIFID with and without the BCSCN. In comparison to the disparity observed without BCSCN processing, the distributional gap of "clean" images has significantly diminished, resulting in lower SIFID scores. Figure 3 presents qualitative outcomes, showcasing that the reconstructed "clean" images not only retain essential content and intricate details but also effectively eliminate residual structures, noise, and compression artifacts. These findings show BCSCN significantly reduces the distributional disparities between training and testing images.To assess the effectiveness of our method, we introduce two distinct implementations of the BCSCN-SR: BCSCN-ESRGAN and BCSCN-PASD 𝑏 . BCSCN-ESRGAN combines BCSCN with the classical GANbased SR method ESRGAN [34] . Considering the potent generative capabilities of diffusion models, we also present a version utilizing diffusion techniques in the hope of achieving enhanced performance. BCSCN-PASD 𝑏 integrates BCSCN with the Pixel-Aware Stable Diffusion (PASD) model [40] , an SR approach based on stable diffusion (SD) [26] . We solely adopted its network architecture, omitting the degradation supervision component, and fine-tuned the model starting from SD. For training the SR network, we kept the parameters of BCSCN fixed and used a paired LR-HR dataset synthesized through bicubic degradation from 3,000 HR images in the DF2K (DIV2K [28] + Flickr2K [29] ) dataset as the training set.To validate the BCSCN's ability to enhance the generalization performance of SISR models, we carried out comparative assessments against the following SOTA SR methods that similarly were trained using bicubic downsampling: EDSR [17] , RCAN [43] , NLSA [23] and SwinIR [15] . EDSR represents a classical residual learningbased approach. RCAN, NLSA, and SwinIR employ techniques based on channel attention, non-local attention, and the Swin Transformer, respectively. We also included the ESRGAN and PASD models trained on bicubic downsampling as our baseline models. Except for PASD's bicubic version, PASD 𝑏 , which required additional training, all baseline models utilized their provided pretrained weights. All methods were trained solely on a training set with a bicubic degradation kernel and then directly applied to test sets with unknown degradation kernels.Table 2 presents the quantitative evaluation results, demonstrating that the BCSCN preprocessing network significantly enhances the generalization performance of both ESRGAN and PASD 𝑏 . Specifically, (1) BCSCN-ESRGAN outperforms ESRGAN in LPIPS and FID metrics by 38.0% and 30.5% respectively, while BCSCN-PASD 𝑏 surpasses PASD 𝑏 by 32.2% and 29.4% in the same metrics, affirming the efficacy of BCSCN in boosting the generalizability of SR models. (2) BCSCN-PASD 𝑏 exhibits further advancements in generalization performance compared to BCSCN-ESRGAN, with improvements of 7.2% in LPIPS and 17.8% in FID. This improvement can be attributed to the rich natural image priors within the stable diffusion model, which provide extensive texture details, albeit at the cost of a slight reduction in pixel consistency compared to BCSCN-ESRGAN. (3) Both BCSCN-PASD 𝑏 and BCSCN-ESRGAN significantly outperform other baseline methods in terms of overall performance, achieving superior perceptual quality. Furthermore, within the AIM2019Track2 and NTire2020Track1, PSNR and SSIM are not as advantageous, primarily because GAN or diffusion-based approaches can generate more realistic details, albeit with a compromise on pixel consistency.Figure 4 showcases the qualitative results. The image '0821' from Ntire2018Track2 suffers from severe blurring and noise, making it a challenging task for most baseline methods to effectively restore such images. This is particularly true for ESRGAN, which tends to exacerbate noise, leading to severe artifacts. In contrast, our approach, whether BCSCN-ESRGAN or BCSCN-PASD 𝑏 , produces images with clear and sharp edges. Moreover, while baseline methods can partially restore the edges of the slightly degraded image '0803', they inevitably introduce intolerable artifacts. However, our methods are capable of generating clean and sharp images, with BCSCN-PASD 𝑏 being notably effective in restoring the edges of petals and generating surprisingly realistic details.Dataset. Similar to the real-world superresolution challenge datasets, as illustrated in Table 2 , our method achieved the highest NIQE and MUSIQ scores on the RealSRSet, with BCSCN-PASD 𝑏 surpassing the top-ranked baseline method, PASD 𝑏 , by 19.4% and 30.2% in NIQE and MUSIQ, respectively. Qualitatively, as shown in Figure 5 , unlike baseline methods introduce significant high-frequency artifacts, our method maintains robustness against real degradation, restoring clear and clean edges.In addressing the real-world SR task, besides the method proposed in this paper, there is another popular category of methods known as blind SR methods. Among these methods, many of the bestperforming ones commonly adopt second-order degradation kernels. The term second-order refers to the image degradation process being divided into two consecutive stages. In each stage, a classic degradation model with adjustable parameters is randomly selected to degrade the image. This type of degradation model aims to cover a wide range of degradation scenarios encountered in real-world applications by extensively sampling various combinations of various types and parameters of classic degradation models as much as possible. While these methods, such as RealESRGAN [33] , and diffusion-based methods like StableSR [31] , DiffBIR [18] , and PASD [40] , exhibit excellent super-resolution capabilities on artificially degraded data, their performance tends to diminish when applied to real-world scenes where human knowledge of the degradation kernel is lacking. This can be observed from Table 3 , where their performance leads on the artificially synthesized datasets RWSRC, but on the real-world dataset RealSRSet, their performance is not as outstanding. The quantitative results of RWSRC in the table are the average across the four RWSRC datasets. Compared to SOTA blind SR methods, the approach presented in this paper strives to minimize distribution discrepancies and enhance the generalization capability of subsequent SR networks. This is achieved through the proposed preprocessing network BC-SCN, which is trained solely on a single, simple bicubic kernel without incorporating any additional degradation. This makes the method less sensitive to changes in the degradation model, and it achieves satisfactory results on both artificially synthesized datasets RWSRC and real-world dataset RealSRSet. Notably, on the Real-SRSet, BCSCN-PASD 𝑏 achieves first rank in NIQE and second in MUSIQ, as evidenced by Figure 6 , where its ability to recover clear frog legs and branches alongside realistic textures results.Furthermore, it should be pointed out that, compared to these leading blind super-resolution methods, BCSCN-PASD 𝑏 has a higher sample utilization rate and requires fewer resources during training.For training, RealESRGAN, StableSR, and PASD utilized between 13K to 25K HR images from various datasets, whereas DiffBIR was To evaluate the impact of various components of BCSCN on the performance of BCSCN-SR, we build upon the foundation of BCSCN-ESRGAN to conduct a comprehensive analysis across four dimensions: patch size, number of bases, bézier curve basis order, and reward functionality. The overall performance was assessed using average metrics computed across four RWSRC datasets. decomposed into 50 bases. Results reveal a diminishing trend in indicators with increasing patch size. Figure 7 visually demonstrates that larger patch sizes amplify image complexity, challenging the agent in reconstructing intricate details and resulting in smoother SR outcomes. Performance convergence is observed when the patch size falls below 16, with nuances in detail becoming less conspicuous. Table 5 elucidates the influence of the number of bases on SR results. Maintaining a fixed patch size decomposition at 16 × 16, we conduct a comparative analysis for four distinct base quantities, ranging from low to high. Quantitative metrics show swift improvement when the base quantity is below 20, plateauing beyond 20 bases. Figure 7 qualitatively supports this observation, with incremental basis enhancing window pattern intricacies. The transition from 5 to 20 bases manifests a notable escalation in detail, while minimal variation is observed from 20 to 30 bases. The detailed process of decomposing LR images is available in the Appendix. 4.7.2 Bézier Curve Basis Order Analysis. Table 6 demonstrates the impact of utilizing Bézier curve bases of different orders on performance. As the order of the bases increases, there is no significant change in model performance, indicating that due to the lower complexity of structures within small image patches, a second-order Bézier curve is sufficient for reconstructing the LR images. improve, affirming each function's effectiveness. Figure 8 shows that when guiding the sparse encoding agent solely with the WGAN reward, clean edge structures are restored but introduce artifacts. As the regional reward is enhanced, these artifacts diminish, albeit with a reduction in image texture details. However, by introducing additional Laplace rewards, lost details are effectively recovered, resulting in a high-quality SR output that is both clean and sharp.  
 
 Appendix:  


Decision: This paper received mixed scores in pre-rebuttal review round: 1 Weak Reject, 1 Borderline Accept, 2 Weak Accept. Thanks to the nice rebuttal, two reviewers raised scores (from Weak Reject to Borderline Reject, and from Weak Accept to Accept). Finally, it seems all reviewers are postive on this paper, including the reviewer rating Borderline Reject was mostly satisfied with the feedbacks in rebuttal. AC believes that this paper can be accepted, and the authors have to incorporate the core items in rebuttal into the main paper when preparing the camera ready.

Reviews: review: paper_summary: paper_summary: The paper introduces the Bézier Curve basis-based Sparse Coding Network (BCSCN) to improve Single Image Super-Resolution (SISR) by mitigating domain gaps caused by diverse degradation kernels. BCSCN removes visual defects in low-resolution images and uses rewards to maintain main content while eliminating degradation effects. Experiments show that BCSCN enhances the generalization of super-resolution networks.
strength_weakness: strength_weakness: This paper attempts to achieve Image Super-Resolution using a different approach.
limitations: limitations: The comparison with Bézier Curve Sparse Coding in the paper could be expanded to help readers better understand this concept.

In Table 2, the paper only compares with methods like ESRGAN, SwinIR, and PASD, which are not very recent works. It is suggested to include comparisons with more recent methods and networks.

The paper states, "solely on a single, simple bicubic kernel, this makes the method less sensitive to changes in the degradation model." Why would training only on a bicubic kernel make the method less sensitive to changes in the degradation model? Could this method be trained on second-order degradation types? If so, what would the results look like?
paper_summary: paper_summary: This paper aims at narrowing the domain gap between training and testing sets, thus improving the generalization capability of SR models. Specifically, it proposes a Bézier curve basis-based sparse coding preprocessing network BCSCN to solve the distribution disparities between training and testing phases. The underlying principle is to preserve the image content as well as eliminate degradation-related defects (e.g., artifacts, residual structures, and noise), so as to produce “clean” images for subsequent SR operations.
strength_weakness: strength_weakness: Novelty. Moderate original.

Clarity. Well organized and easy to understand.

Technical Correctness. Probably correct.

Reference to Prior Work. References adequate.
limitations: limitations: Inadequate rationalization of experimental settings. According to the authors, BCSCN can be considered as a pre-processing process for SR restoration models. Therefore, it should be compared with solutions with similar ideas or operations. For example, CinCGAN [1] and MCinCGAN [2] can be regarded as representative two-stage restoration algorithms, where the first stage is mainly aimed at obtaining clean LR images, while the second stage is mainly responsible for SR reconstruction of clear LR images. This manuscript should take such studies into account.

[1] Yuan Y, Liu S, Zhang J, et al. Unsupervised image super-resolution using cycle-in-cycle generative adversarial networks[C]//Proceedings of the IEEE conference on computer vision and pattern recognition workshops. 2018: 701-710.

[2] Zhang Y, Liu S, Dong C, et al. Multiple cycle-in-cycle generative adversarial networks for unsupervised image super-resolution[J]. IEEE transactions on Image Processing, 2019, 29: 1101-1112.

Limited experimental validation. The authors used only four real-world SR challenge (RWSRC) datasets to validate the effectiveness of the proposed method. First of all, these datasets are poorly accessible unless the authors can provide their download links. In addition, for more fair comparison, there are already some recognized realistic SR benchmarks, such as RealSR [3], DRealSR [4], DPED [5], etc., which can be used to demonstrate the strong generalization ability of practical SR solutions.

[3] Cai J, Zeng H, Yong H, et al. Toward real-world single image super-resolution: A new benchmark and a new model[C]//Proceedings of the IEEE/CVF international conference on computer vision. 2019: 3086-3095.

[4] Wei P, Xie Z, Lu H, et al. Component divide-and-conquer for real-world image super-resolution[C]//Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part VIII 16. Springer International Publishing, 2020: 101-117.

[5] Ignatov A, Kobyshev N, Timofte R, et al. Dslr-quality photos on mobile devices with deep convolutional networks[C]//Proceedings of the IEEE international conference on computer vision. 2017: 3277-3285.
paper_summary: paper_summary: The authors propose the Bézier Curve basis-based Sparse Coding Network (BCSCN), which is a preprocessing network for extracting clean subjects from input images containing various types of degradation. BCSCN has adopted a reinforcement learning framework and three different types of reward function to produce the optimal basis coefficients within a predefined Bézier Curve basis space.
strength_weakness: strength_weakness: 1.The authors propose a Bézier Curve basis-based Sparse Coding Network. The network skillfully combines a sparse coding framework with reinforcement learning in the preprocessing step, while keeping the super-resolution model unchanged.  
2.The authors designed a reward function with three components, focusing on the trade-off between the reconstruction distance of WGAN, a larger curve area and rich details.  
3.The experimental results indicates BCSCN could improve the current SOTA models.
limitations: limitations: 1.The authors decompose the input image into consecutive small patches for individual reconstruction but do not employ any post-processing mechanism to maintain consistency across the image as in other traditional sparse coded super-resolution models. The inconsistency is evident in Figure 3.  
2.Insufficient experimentation. The four RWSRC datasets used were from the same 100 images of natural scenes, only with different degradation. The RealSRSet dataset is only 20 images. It is unable to show the model's ability to process other types of images.   
3.The number of BCSCN iterations and the additional inference time due to BCSCN are not shown in the experiments.
paper_summary: paper_summary: In summary, this article presents BCSCN, a preprocessing network based on the Bézier curve basis, effectively addressing domain gap challenges by minimizing input distribution discrepancies between training and testing phases. The article introduces a set of rewards to guide the basis coefficients search agent in capturing main content and eliminating degradation-related information. Integration of BCSCN into GAN-based and diffusion-based SR networks, resulting in BCSCN-ESRGAN and BCSCN-PASD𝑏, respectively, yields significant improvements in naturalness performance on real-world datasets with unknown degradation kernels compared to their base models. Moreover, BCSCN-PASD𝑏 achieves competitive results to state-of-the-art blind SR methods while requiring fewer samples and lower training resources.
strength_weakness: strength_weakness: This article introduces several noteworthy contributions. Firstly, BCSCN, a preprocessing network utilizing the Bézier curve basis, effectively reduces input distribution discrepancies between training and testing phases, addressing the domain gap challenge in subsequent SR networks. Secondly, the article presents a set of rewards aimed at guiding the basis coefficients search agent to capture main content and eliminate degradation-related information. Thirdly, the integration of BCSCN into GAN-based and diffusion-based SR networks, resulting in BCSCN-ESRGAN and BCSCN-PASD𝑏 respectively, leads to significant enhancements in naturalness performance on real-world datasets with unknown degradation kernels compared to their base models. Moreover, BCSCN-PASD𝑏 demonstrates competitive results against state-of-the-art blind SR methods while requiring fewer samples and lower training resources.
limitations: limitations: While the proposed BCSCN effectively addresses distribution disparities in both training and testing phases, there are certain aspects that warrant clarification. In the ablation study on different patch sizes (Section 4.7), the authors suggest that larger patch sizes amplify image complexity, challenging the reconstruction of intricate details and resulting in smoother SR outcomes. This raises two important questions: 1) How would the performance be affected if a patch size of 4x4 is used? 2) Following the authors' reasoning, could increasing the Basis Number mitigate the challenges posed by larger patch sizes? If not, are there alternative explanations for this phenomenon?