DDR: Exploiting Deep Degradation Response as Flexible Image Descriptor

Authors: juncheng  wu
zhangkai  ni
hanli  wang
wenhan  yang
yuyin  zhou
shiqi  wang
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-13 12:29:08

Abstract: Image deep features extracted by pre-trained networks are known to contain rich and informative representations. In this paper, we present Deep Degradation Response (DDR), a method to quantify changes in image deep features under varying degradation conditions. Specifically, our approach facilitates flexible and adaptive degradation, enabling the controlled synthesis of image degradation through text-driven prompts. Extensive evaluations demonstrate the versatility of DDR as an image descriptor, with strong correlations observed with key image attributes such as complexity, colorfulness, sharpness, and overall quality. Moreover, we demonstrate the efficacy of DDR across a spectrum of applications. It excels as a blind image quality assessment metric, outperforming existing methodologies across multiple datasets. Additionally, DDR serves as an effective unsupervised learning objective in image restoration tasks, yielding notable advancements in image deblurring and single-image super-resolution. Our code is available at: https://github.com/eezkni/DDR.

Main paper: 
 introduction: 
 Deep features extracted by pre-trained neural networks are well-known for their capacity to encode rich and informative representations [1] [2] [3] [4] . Extensive research efforts have aimed to quantify the information encoded within these deep features for use as image descriptors. For example, the distance between deep features has been employed as a metric for image quality assessment (IQA) in various studies [3, 5, 6] . Additionally, researchers have studied differences between images by comparing the distributions [2, 7] or frequency components [8, 9] of their deep features. Moreover, the statistical properties of deep features have been found to correlate with the style and texture of images in prior works [6, 10] . Recent research has also highlighted that the internal dissimilarity between deep features at different image scales can serve as a potent visual fingerprint [11] .This paper delves into an intriguing and unexplored property of image deep features: their response to degradation. Specifically, when subjecting images with diverse content and textures to various types of degradation, such as blur, noise, or JPEG compression, the deep features of these images exhibit varying degrees of change. This phenomenon is illustrated in Fig. 1 , where Gaussian Blur is applied to images. The degrees of changes in feature space reflect the deep feature response to specific degradation. As shown in Tab. 1, a strong correlation exists between this response and the quality scores of blurred images. We validate that the response of deep features to degradation effectively We apply the same level of Gaussian Blur to different images from the LIVEitw [14] dataset. x and x d denote the original and degraded images, respectively. l(•, •) is the LPIPS metric [3] between x and x d , which measures the extent of changes in the feature space. The results demonstrate that images with different content and texture characteristics exhibit varying degrees of change.Metric SRCC ILNIQE [15] 0.915 WaDIQaM [16] 0.938 BIECON [17] 0.956 HOSA [18] 0.954 DBCNN [19] 0.935 HyperIQA [20] 0.926 DDR blur 0.988Table 1 : Comparison of SRCC for blur degradation on the LIVE [21] dataset. DDR blur refers to the deep feature response to blur obtained by manually synthesizing degradation in the pixel domain.DDR blur demonstrates highest correlation with human opinion.Figure 2 : Distribution of DDR on the LIVEitw [14] dataset. "Low", "optimal", and "high" refer to different levels of handcrafted degradation applied in the pixel domain, while "adaptive" represents adaptively fusing textdriven degradation in the feature domain. The DDR with "optimal" and "adaptive" degradation achieve significantly better performance on the BIQA task.captures different image characteristics by varying the type of degradation. Therefore, we propose the Deep Degradation Response (DDR), which quantifies the response of image deep features to specific degradation types, serving as a powerful and flexible image descriptor.One straightforward approach to compute the DDR is to apply handcrafted degradation to the image, extract degraded features from the degraded image, and then calculate the distance between these degraded features and the features of the original image. However, as shown in Fig. 2 , adjusting the level of degradation applied to the image significantly affects the distribution of DDR. Therefore, meticulous adjustment of the degradation level is crucial to achieve optimal performance in various downstream tasks. To address this challenge, we propose a text-driven degradation fusing strategy. Inspired by manifold data augmentation algorithms [12, 13] , we adaptively fuse text features representing specific degradation onto the original image features, resulting directly in degraded features. By manipulating the text, we can effectively control the type of degradation fused to the image features. This approach allows us to flexibly assess the response of image features to various degradation types, thereby enhancing adaptability across different downstream tasks.We evaluate the performance of our proposed DDR across multiple downstream tasks. Firstly, we assess its effectiveness as an image quality descriptor on the opinion-unaware blind image quality assessment (OU-BIQA) task, where DDR demonstrates superior performance compared to existing OU-BIQA methods across various datasets. Secondly, we employ DDR as an unsupervised learning objective, training image restoration models specifically to maximize the DDR of the output image, which includes tasks such as image deblurring and real-world single-image super-resolution.Incorporating DDR as an external training objective consistently improves performance in both tasks, highlighting the strength of DDR as a flexible and powerful image descriptor. 
related works: 
 Deep Feature Based Image Descriptors. Image descriptors aim to quantify fundamental characteristics of images, such as texture [22] , color [23, 24] , complexity [25] , and quality [26, 27] . With the informative representations in deep features extracted by pre-trained networks, various efforts have been made to develop image descriptors based on these features. Many existing descriptors regress the deep features of an image to a score, training the model by minimizing the loss between these predicted scores and the ground truth scores labeled by humans [25, 20, 28] . However, these methods are somewhat inflexible for two reasons: (1) they rely on human-labeled opinion scores, and (2) they are designed to evaluate fixed image characteristics. In this paper, we propose a flexible alternative by measuring the degradation response of deep features.Image Degradation Representation for Image Restoration. Various deep learning-based image restoration methods leverage image degradation representation to enhance model performance [29] [30] [31] [32] . For instance, some methods utilize contrastive-based [29] and codebook-based [30] techniques to encode various degradation types, enhancing the model's robustness to unknown forms of degradation. Moreover, other methods design degradation-aware modules to extract degradation representations from images and guide the removal of degradation [31, 32] . However, these methods depend on task-specific training. In contrast, the proposed DDR flexibly obtains representations for different types of degradation, making it an effective image descriptor for various image restoration tasks.Recent advancements in multimodal vision models, achieved through extensive training on paired image-text data [33] [34] [35] [36] [37] , have significantly enhanced the capability of these models to understand and describe image textures using natural language [38] . Researchers have explored various methods to leverage this capability for modifying image texture attributes through language guidance. For instance, in language-guided image style transfer [39, 40] , natural language descriptions are used to define the target texture style. Additionally, Moon et al. [12] introduced a manifold data augmentation technique that integrates language-guided attributes into image features.Building on these ideas, our work aims to adaptively fuse degradation information into image deep features using language guidance, thereby facilitating the measurement of our proposed DDR.3 Deep Degradation Response as Flexible Image Descriptor 
deep degradation response: 
 We define the Deep Degradation Response (DDR) as the measure of change in image deep features when specific types of degradation are introduced, which can be mathematically expressed as:EQUATIONwhere d represents the type of degradation. M (•, •) denotes a disparity metric, such as L n distance or cosine distance. F = Φ v (i) represents the original image features extracted by a pre-trained network Φ v (•), while F d denotes the degraded features.The core of the proposed DDR lies in how to model F d . A naive approach to this involves synthesizing degradation in the pixel domain, i.e., generating degraded images. As shown in Fig. 3 (a), a handcrafted degradation process is applied to the image, leading to the creation of a degraded image i d . The extent of degradation is controlled by the parameter ω d . Then a pre-trained visual encoder Φ v (•) is utilized to extract the features of i d , generating degraded features. Therefore, the pixel space degradation synthesizing process can be formulated as:EQUATIONwhere D(•) denotes the handcrafted degradation synthesis process. However, as depicted in Fig. 2 , varying levels of degradation significantly affect DDR. For downstream tasks, it is imperative to meticulously determine the optimal ω d for different manual processes. This not only poses a substantial challenge but also diminishes the robustness of DDR as an image descriptor.EQUATIONIn this study, we propose a novel and efficient method for modeling F d by synthesizing degradation in the feature domain using text-driven prompts. Specifically, to construct the degradation representation, we first design a pair of prompts: one describing an image with a specific type of degradation and the other describing the same image without degradation. These prompts are then separately encoded using the text encoder Φ t (•) of CLIP [37] , yielding text-driven degradation representations T - d and T + d , respectively. We obtain the degradation direction in the feature space by calculating the difference between these representations, as follows:EQUATIONwhere T - d = Φ t (P - d ) and T + d = Φ t (P + d ). P - d and P + d represent the degraded and clean prompts, respectively. However, due to the gap between text and image modality within the feature space [41, 42] of the CLIP model, we cannot effectively obtain the degraded image feature by directly fusing the features from different modalities. To address this challenge, we propose an adaptive degradation adaptation strategy by 'stylizing' the text-driven degradation representation using the image feature. Inspired by AdaIN [43] , we propose to align the mean and variance of T d to match those of the image feature, which can be formulated as follows:EQUATIONwhere Td denotes the adapted degradation representation. Finally, we fuse the image feature with Td , and the feature space text-driven degradation process can be represented as:EQUATIONOur proposed degradation fusion method allows us to measure DDR across various types of degradation simply by modifying the text prompt, eliminating the need for handcrafted design processes. Additionally, our adaptation strategy enables the application of text-driven degradation to image features without adjusting any hyper-parameters. As shown in Fig. 2 , in the LIVEitw [14] dataset, DDR with text-driven feature degradation method achieves a distribution similar to DDR with carefully adjusted optimal degradation level, demonstrating the flexibility of our method.By modifying the degradation type, DDR can capture different characteristics in natural images. We demonstrate this by measuring DDR with different degradation types across all images in the We measure DDR with five types of degradation by setting their corresponding prompt pair. We observe that image with lower DDR to a specific type of degradation is likely to obtain this degradation of a higher level. LIVEitw [14] dataset. Specifically, we set five pairs of prompts, representing five types of degradation, including color, noise, blur, exposure, and content. We employ a fixed prompt formatting for different types of degradation, as follows:EQUATION)For example, when the degradation type is blur, the d -and d + are set as 'blurry' and 'sharp' respectively. The images with high and low DDR for each type of degradation are shown in Fig. 4 . We observe a negative correlation between the DDR and the level of degradation within the image. For example, as demonstrated in Fig. 4 (e), an image with a high DDR to content degradation retains clear content, while the corresponding image with a low DDR exhibits unrecognizable content.To further quantify the correlation between DDR and other image characteristics, we calculate the Spearman's Rank Correlation Coefficient (SRCC) between DDR and four types of image characteristics. Specifically, we measure the complexity [25] , colorfulness [23] , and sharpness [44] of images in the CSIQ [45] dataset. Additionally, we use the Mean Opinion Score (MOS) of each image as its quality score. The results are presented in Tab. 2. It is interesting to note that there is a negative correlation between the complexity of an image and DDR. This suggests that more complex images are capable of enduring more degradation with a smaller degree of change in deep features. Furthermore, the DDR to color and blur degradations show the highest correlation with colorfulness and sharpness respectively. Overall, with different degradation types, DDR tends to emphasize different image characteristics. Therefore, DDR shows promise as a versatile image descriptor for diverse downstream tasks through simple prompt adjustments, including IQA and image restoration.DDR can function as an image quality descriptor. As shown in Tab. 2, there is a positive correlation between the quality score and the DDR of the image. In cases where image quality is predominantly affected by a specific degradation type, an image with a high DDR to this degradation would likely obtain a higher quality. For instance, in Fig. 4 , when the degradation type is blur, comparing the image with a high DDR to the image with a low DDR, it is evident that the former exhibits a sharper content with less blur. However, real-world images often feature a mix of degradations. To evaluate image quality in such scenarios, we formulate a set of degradations denoted as D and compute the mean DDR for each degradation in D. The blind image assessment metric based on DDR can thus be formulated as follows:EQUATIONWe can also utilize DDR as a learning objective in image restoration tasks, where the goal is to train a deep learning-based restoration model to predict a clean image from a degraded one. This is achieved by optimizing the restoration model to minimize the reconstruction loss function, which quantifies the difference between the pixel values of the model's output and the ground truth. In this work, we demonstrate that incorporating DDR as an external unsupervised learning objective can improve the optimization of the restoration models. Specifically, we measure the DDR of the model output and aim to simultaneously minimize the reconstruction loss while maximizing the DDR. The learning objective of the image restoration model is thus formulated as:EQUATIONwhere R θ (•) is a restoration model parameterised by θ, and L rec (•, •) denotes the reconstruction loss, λ d is the weight of DDR in learning objective. Similarly, by adjusting the degradation prompt and combining different types of degradation, we can tailor the approach to various restoration tasks. Implementation Details. For different tasks, we tailor the degradation set D in Eq. 7 and Eq. 8 to focus on distinct image attributes. Specifically, in BIQA, we define D = {color, noise, blur, exposure}. Meanwhile, for image restoration tasks, we set D = {color, content, blur}. In all experiments related to image restoration, we empirically set the weight of DDR in the learning objective in Eq. 8 as λ d = 2.0. We use the CLIP [37] ViT-B/32 model as the image feature extractor, and employ the cosine distance to quantify the disparity between original and degraded image features, which can be defined as follows:EQUATIONBaseline Datasets. To evaluate the effectiveness of the proposed DDR as an image quality descriptor, we conduct extensive experiments on eight public IQA datasets, including CSIQ [45] , TID2013 [46] , KADID [47] , KonIQ [48] , LIVE in-the-wild [14] , LIVE [21] , CID2013 [49] , and SPAQ [50] , which encompass both synthetic and real-world degradation scenarios. For image deblurring, we train and test the model using the GoPro dataset [51] and RealBlur dataset [52] , respectively. The GoPro dataset [51] consists of synthetic blurred images, while RealBlur [52] contains images with real-world motion blur. For SISR, we combine two real-world datasets together for training and testing, including the RealSR [53] and City100 [54] datasets.Baseline Methods. For the OU-BIQA task, we compare DDR with representative and state-of-the-art opinion-unaware BIQA (OU-BIQA) methods, which do not require training with human-labeled MOS. The compared methods include NIQE [26] , QAC [55] , PIQE [56] , LPSI [57] , ILNIQE [15] , diqIQ [58] , SNP-NIQE [59] , NPQI [60] , and ContentSep [61] . Among all compared methods, DDR is the only zero-shot method that does not require any training. For image restoration, we compare our proposed method, as illustrated in Eq. 8, with a combination of reconstruction loss and feature domain loss L f (•, •), which quantifies the distance between deep features extracted from images. Generally, the reconstruction loss is combined with feature domain losses to enhance the overall quality of the restored image, forming the learning objective of the restoration model R θ (•) as follows:EQUATIONwhere λ f is the weighting factor for L f (•, •). In all experiments on image restoration, we utilize PSNR loss as the reconstruction loss. We consider four types of representative feature domain losses for comparison, including LPIPS [3] , CTX [2] , PDL [7] , and FDL [8] . To ensure a fair comparison, we set λ f = 0.1 for FDL [8] and λ f = 1.0 for the other feature domain losses, ensuring that the magnitudes of the different feature domain losses are in a similar range.Moreover, to fully assess the robustness of our proposed DDR across vaious architectural models, we conduct all image restoration experiments using two representative image restoration models: NAFNet [62] and Restormer [63] . NAFNet [62] is a convolutional neural network (CNN)-based model, while Restormer [63] is a Transformer [64] -based model. These models have demonstrated impressive performance in their respective tasks and are widely recognized as representative models in recent years. We empirically train the model at a resolution of 128 × 128. For the learning rate, we adhere to the official settings for NAFNet and Restormer. Specifically, the initial learning rate for NAFNet is set to 1e -3, and for Restormer, it is set to 3e -4. We also adopted a cosine annealing strategy for both models. 
opinion-unaware blind image quality assessment: 
 Tab. 3 presents the results across all datasets. Our proposed DDR consistently outperforms all competing methods on datasets with both synthetic [45] [46] [47] and in-the-wild [14, 49, 48, 50] degradation, underscoring its robustness across diverse degradation types. Especially its substantial improvement in SRCC on the LIVE in-the-wild dataset, rising from 0.5060 to 0.6613, showcasing the effectiveness of DDR as an image quality descriptor for images with real-world degradation. Furthermore, comparing the SRCC performance in Tab. 3 and Tab. 2, it is obvious that on the CSIQ dataset [45] , DDRs that integrate multiple types of degradation perform significantly better than DDRs that only focus on a single type of degradation. This underscores the superiority of DDR as a more comprehensive image quality descriptor simply by integrating multiple types of degradation.The objective of image deblurring is to restore a high-quality image with clear details. The quantitative analysis in Tab. 4 illustrates that our proposed DDR surpasses all compared loss functions across datasets with both synthetic and real-world blur. Compared to optimizing solely the PSNR loss, our approach achieves a notable enhancement in PSNR, with an increase of at least 0.16 dB across all models and datasets. These results suggest that maximizing the DDR of the predicted image results in higher fidelity and reduced degradation. This is further evident in the qualitative results shown in Fig. 5 , where the PSNR loss alone produces blurry textures, and combining PSNR with feature [51] and real-world [52] blur respectively. The best results are marked in bold. Combining proposed DDR with reconstruction loss leads to result with less degradation and higher fidelity. Our proposed method demonstrates the robustness to model architecture and dataset.domain losses introduces noticeable artifacts. In contrast, incorporating DDR substantially reduces artifacts, yielding predicted images with sharper and more natural textures.SISR is a task aimed at enhancing the resolution of a low-resolution image to match or surpass the quality of a high-resolution counterpart. In our study, we evaluate our proposed DDR method against state-of-the-art loss functions. Tab. 5 showcases the quantitative results on a real-world dataset by two representative models (NAFNet and Restormer). Our findings reveal that our method outperforms all competing methods in terms of PSNR. Particularly noteworthy is the improvement achieved with NAFNet, where the incorporation of DDR alongside the reconstruction loss elevates the PSNR from 27.08 to 27.31. Additionally, as depicted in Fig. 6 , our method yields visual results with finer texture compared to those optimized solely for PSNR or combined with LPIPS. For image deblurring, we conduct a series of ablation experiments on the NAFNet using the GoPro dataset, with all results detailed in Table 6 . Firstly, we adjusted D to evaluate the effect of the degradation set defined in Eq. 8. Notably, a decrease in performance is observed when any type of degradation is removed, suggesting that combining multiple types of degradation results in a more comprehensive image description. Secondly, we investigate the effect of λ d in Eq. 8. Minor fluctuations in performance are observed when adjusting λ d to 1.0 and 3.0, indicating the robustness of our method to this hyper-parameter. Next, we explore the effect of the visual feature extractor.Increasing the scale of Φ v in DDR does not lead to improved performance, suggesting that a larger visual model may not necessarily enhance the ability to understand low-level texture. Finally, we examine the impact of the adaptation strategy in Eq. 4. A significant drop in performance is observed when the adaptation is removed, highlighting the critical role of this strategy in DDR calculation.For opinion-unaware blind image quality assessment task, we conduct ablation experiment on four datasets. As demonstrated in Tab. 7, comparing with measuring DDR to single type of degradation, combining mutiple degradation types consistently leads to significant performance improvement. Furthermore, we can observe a performance boost by utilizing degradation adaptation strategy, improving SRCC from 0.6074 to 0.8289 on CSIQ dataset. 
limitations and discussion: 
 In this section, we discuss the limitations of DDR and provide potential solutions to address them.The ability of the visual feature extractor to understand low-level degradation. We currently employ the CLIP model's visual feature extractor to facilitate text-driven degradation fusion. These feature extractors may incline to focus on high-level information such as image content, while their ability to understand low-level degradation may be limited. This could impact the measurement of the degradation response. In future work, we plan to fine-tune the feature extractor on tasks such as degradation classification or description, to enable it to extract more fine-grained degradation features.The selection of degradation prompts for different downstream tasks. The suitable degradation prompts may vary for different downstream tasks. In future work, we hope to append learnable tokens in the degradation prompts, and fine-tune these tokens to better adapt our method to different tasks. Specifically, we can utilize the strategy such as adversarial training, training DDR as a discriminator. This is an interesting and promising direction for our further investigation.Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material? Answer: [Yes] Justification: We will release our code after the paper is accepted. Guidelines:• The answer NA means that paper does not include experiments requiring code. • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 
conclusions: 
 This paper introduces a flexible and powerful image descriptor, which measures the response of image deep features to degradation. We propose a text-driven approach to adaptively fuse degradation into image features. Experimental results demonstrate that DDR achieves state-of-the-art performance in blind image quality assessment task, and optimizing DDR results in images with reduced distortion and improved overall quality in image restoration tasks. We believe that DDR can facilitate a better understanding and application of image deep features. interesting to investigate the correlation between DDR and deep feature statistics. Specifically, we extract features from five layers (Relu_1_1, Relu_2_1, Relu_3_1, Relu_4_1, and Relu_5_1) of pretrained VGG [67] network, and measure the mean and standard deviation of extracted features. Then, we utilize SRCC to quantify the correlation between DDR and these statistics. The results are shown in Fig. 7 . We can observe that DDR to color degradation demonstrates similar correlation to statistics of feature from every layers. While for exposure and blur degradation, DDR shows significant higher correlation to mean and standard deviation of feature from Relu_1_1 than subsequent layers. In contrast, for noise and content degradation, DDR shows higher correlation for Relu_1_1 and Relu_5_1.Secondly, we present additional qualitative results for image restoration tasks. Fig. 9 and Fig. 8 show the results in image deblurring on the realBlur [52] dataset and GoPro [51] dataset respectively. Moreover, the qualitative results in SISR on real-world dataset [53, 54] are demonstrated in Fig. 10 . Figure 10 : Qualitative result of SISR using the NAFNet [62] trained with real-world SISR [53, 54] dataset. The red area is cropped from different results and enlarged for visual convenient.• The answer NA means that the paper does not include experiments.• If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not. • If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable. • Depending on the contribution, reproducibility can be accomplished in various ways.For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed. • While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. , with an open-source dataset or instructions for how to construct the dataset). (d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility.In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes] Justification: We provide all the training and test details in the main paper and supplementary material.Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 
experiment statistical significance: 
 Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [No] Justification: Our paper does not report error bars.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper. • The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions). • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors). • It should be clear whether the error bar is the standard deviation or the standard error of the mean. • It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified. • For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates). • If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text. 
experiments compute resources: 
 Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?Answer: [Yes] Justification: Our paper provides sufficient information on computer resources in supplemental material.Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage. • The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute. • The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper). 
code of ethics: 
 Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines? 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed? 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)? Answer: [NA] Justification: The paper does not involve crowdsourcing nor research with human subjects. Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: Justification: The research conducted in the paper conforms with the NeurIPS Code of Ethics.Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).Justification: There is no societal impact of this work because our work is a foundational research.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 


Decision: The final scores for this work are two borderline accept and one accept. All reviewers showed a generally positive attitude towards the work, with no significant disagreements. After the authors provided their rebuttal, all reviewers clearly indicated that their previous concerns had been addressed and chose to upgrade their ratings. In summary, I am inclined to accept this work.

Reviews: review: paper_summary: paper_summary: In this paper, the authors propose a feature descriptor to assess low-level image quality degradations. Based on CLIP, the proposed method first encodes input image and its degraded version to features in CLIP space; the input image is encoded by CLIP image encoder, and the degraded image feature is generated by adding a textual feature of degradation to the image feature. The Deep Degradation Response (DDR) is measured by calculating the distance (seems that cosine is used) of two features. The authors demonstrated the effectiveness of the proposed descriptor with extensive experiments and analysis including SRCC test and applications to image deblurring and super-resolution.
strength_weakness: strength_weakness: 1. Clarity: Some technical details are unclear, which limited the understanding of the proposed method.
- I did not understand the distribution of DDR in Figure 2. Why is the distribution of "adaptive" better than "low" and "high"? What did the authors do specifically for "optimal"?
- The authors mention in L84-85 that there are options in the disparity metric M, but in SRCC evaluation there exist a clear positive or negative direction of correlation. Since Ln and cosine metrics have different meaning for low/high values, more clarification is needed in the description of the method. It seems that the authors used cosine metric in their code.
- It seems that the method simply used pre-trained CLIP features for both images and texts. Is it correct?
- Among degradation types, Color and Content seem ambiguous. Do the authors have a clear definition of these categories?
2. Even though the DDR can score good and bad quality images in terms of (color, noise, blur, ...), it seems difficult to measure the amount of degradation such as noise level. It makes me wonder if the DDR contains useful information to measure degradations, or simply focus on visually pleasing images because positive texts usually correspond to those images? This is a reasonable question as the authors are aware of the fact that the CLIP features may have bias to high-level features rather than low-level degradations.
3. Continuing 2., although the authors framed the paper as image descriptor for degradations, I think the paper is more relevant to non-reference image quality assessment. Therefore, more prior works in this line of research need to be discussed in Sec. 2.
4. Although it is mentioned in the limitation, the proposed method is evaluated on one type of sentence per each degradation. Can the authors justify why they chose these specific words? Due to the nature of texts, there would be similar words that share similar meanings. How robust is the proposed method against the choice of words?
questions: questions: The paper generally addresses an interesting problem of image quality descriptor using CLIP textual features, and the results look encouraging. However, it lacks the in-depth understanding of the CLIP and proposed method. I would like to hear the authors' answers to the points raised in Weaknesses before adjusting my rating.
limitations: limitations: The authors described two limitations of the proposed method. A potential negative societal impact would be
- The proposed method could be potentially used to discriminate a certain group of photos (e.g. race, gender, etc) by including those words in degradation prompts.
paper_summary: paper_summary: The paper introduces Deep Degradation Response (DDR), a method to quantify changes in image deep features under varying degradation conditions. DDR facilitates flexible and adaptive degradation through text-driven prompts. It reports to excel in blind image quality assessment and image restoration tasks like deblurring and super-resolution. The paper compares the proposed DDR with the existing techniques across multiple datasets. The authors plan to release their code for public use.
strength_weakness: strength_weakness: 1. The performance on BIQA is not particularly competitive, and some of the latest IQA metrics, such as LIQE, UNIQUE, and TreS, are not included for comparison. Please refer to the paper “Blind Image Quality Assessment via Vision-Language Correspondence: A Multitask Learning Perspective”.
2. In Equation 8, DDR is calculated using the restored image and its corresponding degradation. According to the results presented in Table 4, the proposed DDR performs better than PSNR+LPIPS. Since LPIPS directly minimizes the feature difference between the restored image and the original image, it should theoretically be more effective for image restoration. The paper does not explain why DDR outperforms the LPIPS loss. Moreover, in Table 6, the pre-trained model also influences the performance of DDR. Does this imply that the gains achieved by DDR might be due to a stronger backbone in comparison to LPIPS?
3. The paper utilizes DDR as an image quality assessment metric but does not provide an explanation for why it could represent the quality of an image.
questions: questions: Given that the negative and positive values are fixed, T_d)is a constant vector for each degradation. DDR seems to measure the disparity after adding \hat{T_d} to the feature. Does a high DDR indicate that the image distribution is robust to the feature interference caused by T_d?
limitations: limitations: The authors address the limitations and potential societal impact of their work.
paper_summary: paper_summary: This paper proposed a low-level visual descriptor with text embeddings and explored its application on many low-level vision tasks.
strength_weakness: strength_weakness: **Weaknesses of the methods.**

1. This paper uses a text-based model to describe the low-level visual information of the images, which seems to be similar to Q-Bench [1]. I suggest that the author compare it with it and emphasize the differences between the method proposed and Q-Bench.
2. Simple super-resolution networks can identify different low-level visual information [2,3], such as degradation types. The author should compare with this method. Also, the description of lines 69 and 70 could be appropriately modified based on this.
3. The network settings for single-image super-resolution (SISR) are somewhat unreasonable. NAFNet and Restormer have a lot of downsampling and are not suitable as base models for SISR experiments. I suggest to use RCAN, SwinIR, and HAT.

[1] Wu H, Zhang Z, Zhang E, et al. Q-Bench: A Benchmark for General-Purpose Foundation Models on Low-level Vision.

[2] Liu Y, Liu A, Gu J, et al. Discovering distinctive" semantics" in super-resolution networks.

[3] Liu Y, He J, Gu J, et al. Degae: A new pretraining paradigm for low-level vision.
questions: questions: 1. The order of the figures and tables in this paper seems a little confusing and should be more carefully formatted. For example, the experimental results of SISR are not introduced until the end of page 8, but the result images are shown at the beginning of page 8.
2. The experimental settings are not fully given, such as the learning rate and training resolution, which may lead to unfair comparison. For example, SISR models are often trained on 48 $\times$ 48 resolution, are the NAFNet and Restormer also trained on 48 resolution for SISR tasks in this paper?
limitations: limitations: Please see the weaknesses and questions.

Comments: title: Thanks for your comments
Thanks for your comments
Thanks for your comments
Thanks for your comments
comment: Thank you very much for your thoughtful comments and appreciation of our work. We will do our best to improve the final version of our paper based on your valuable suggestions.

Thank you!
Thank the authors for addressing the raised concerns. I have read all reviews and the authors' responses. All of my concerns are resolved and I think the paper has a valid contribution to be published at the conference. I will increase my rating to borderline accept.
Thank you very much for your valuable comments. We hope our responses have addressed your concerns, and we would be happy to respond to any further queries.

Thank you!
Thank you very much for your thoughtful comments and appreciation of our work. We will do our best to improve the final version of our paper based on your valuable suggestions.
This difference shows that this paper contributed to the IQA field. My concerns have been addressed and I will raise the score to accept (7).
We appreciate for your thoughtful comments and insights.

Regarding your concern towards the comparison with Q-bench, we acknowledge that both Q-bench and the proposed DDR approach necessitate alignment between modalities of degradation and text. Specifically, Q-bench explicitly instructs MLLMs to align these modalities, while DDR requires alignment in latent space to facilitate the generation of degradation representations. Your insights have provided a valuable perspective on this issue.
Thanks for authors' response on the comments. I still have some concerns about DDR.

**Comparison with Q-bench.** Could we think that Q-Bench explicitly aligns the two modalities of degradation and text, while DDR aligns these two modalities in latent space?
Thank you very much for your thoughtful comments and appreciation of our work. We will do our best to improve the final version of our paper based on your valuable suggestions.
Thanks for authors' response on the comments. Most of my concerns have been addressed.