Not All Diffusion Model Activations Have Been Evaluated as Discriminative Features

Authors: benyuan  meng
qianqian  xu
zitai  wang
xiaochun  cao
qingming  huang
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-10 06:15:06

Abstract: Diffusion models are initially designed for image generation. Recent research shows that the internal signals within their backbones, named activations, can also serve as dense features for various discriminative tasks such as semantic segmentation. Given numerous activations, selecting a small yet effective subset poses a fundamental problem. To this end, the early study of this field performs a large-scale quantitative comparison of the discriminative ability of the activations. However, we find that many potential activations have not been evaluated, such as the queries and keys used to compute attention scores. Moreover, recent advancements in diffusion architectures bring many new activations, such as those within embedded ViT modules. Both combined, activation selection remains unresolved but overlooked. To tackle this issue, this paper takes a further step with a much broader range of activations evaluated. Considering the significant increase in activations, a full-scale quantitative comparison is no longer operational. Instead, we seek to understand the properties of these activations, such that the activations that are clearly inferior can be filtered out in advance via simple qualitative evaluation. After careful analysis, we discover three properties universal among diffusion models, enabling this study to go beyond specific models. On top of this, we present effective feature selection solutions for several popular diffusion models. Finally, the experiments across multiple discriminative tasks validate the superiority of our method over the SOTA competitors. Our code is available at https://github.com/Darkbblue/generic-diffusion-feature.

Main paper: 
 introduction: 
 Diffusion models [21, 10, 37, 36] are powerful generative models that progressively reconstruct images from Gaussian noises through a series of denoising steps. Typically, a U-Net [38] is trained as the noise predictor backbone to perform denoising. Recently, the impressive generative capability inspires the application to discriminative tasks such as semantic segmentation [52, 58] or semantic correspondence [29, 56] . In this direction, diffusion feature is one simple yet effective approach, where the intermediate signals, named activations, are extracted from the pre-trained diffusion U-Net as dense features [2, 58, 56, 29, 53, 13, 14] .Figure 1 : Prior arts only consider a small fraction of potential activations in diffusion models. As a result, more advanced diffusion architecture fails to achieve better performance (SDXL v.s. SDv1.5). In contrast, we consider a broader range of candidate activations. To facilitate the quantitative comparison, we first make a comprehensive and generalizable analysis to qualitatively filter out many candidates in advance. On top of this, our method achieves superior performance (75.2 PCK@0.1).The complex architecture of the diffusion U-Net provides many activations that can serve as features. However, these activations inherently vary in quality, inducing significant performance gaps on discrimination. Hence, selecting a small yet effective subset from these activations has become a fundamental problem. In the early stage, Baranchuk et. al [2] perform a large-scale quantitative comparison among activations within Guided Diffusion [10] . Later, the activations they select are followed by most studies in this field, pursuing other improvements [29, 56, 58, 52, 60, 53, 24, 27] .However, we find that this fundamental issue is far from solved. On one hand, Baranchuk et. al [2] only consider activations between neighboring modules that comprise the main residuals. This means that many potential activations are excluded from the candidate pool, such as the queries and keys in the self-attention blocks. Moreover, recent developments of diffusion architecture, such as cross-attention [37] or embedded deep vision transformers (ViTs) [36] , have introduced additional types of activations. Hence, as shown in upper Figure 1 , only a small fraction of potential activations in modern diffusion models have been evaluated for their discriminative ability, which might hinder future work in this direction. For example, lower Figure 1 shows that Stable Diffusion XL (SDXL) [36] , which is more advanced than Stable Diffusion v1.5 (SDv1.5) [37] , fails to achieve better performance with the feature selection solution proposed in [2] .In this paper, we revisit the fundamental problem of feature selection, considering a more comprehensive candidate pool of activations. Due to its large volume, a full-scale quantitative comparison is no longer operational, urging us to modify the previous research methodology in [2] . As illustrated in Figure 1 , instead of a direct quantitative comparison, we first explore the properties of diffusion U-Nets. These properties allow us to qualitatively and efficiently filter out many activations that are highly likely to be sub-optimal, shrinking the candidate pool for the quantitative comparison. More importantly, we find these properties are universal among diffusion models, making it possible to generalize our findings to more models beyond those covered in this paper.Specifically, the properties we find, which are distinct to existing knowledge of model properties [1, 38] , exactly correspond to three top-to-bottom levels of the diffusion U-Net: (i) Diffusion noises at the macro level: the diffusion process induces a new type of noise on both low-and high-frequency signals. (ii) In-resolution granularity changes at the in-resolution level: the changes in information granularity are not only across but also within resolutions. (iii) Locality without positional embeddings at the sub-module level: the embedded ViTs in diffusion U-Nets present a new type of local information different from that induced by the conventional positional embeddings [44] . Based on these insights, we develop effective feature selection solutions for several popular diffusion models. Finally, the experiments on three discriminative tasks, including semantic correspondence, semantic segmentation, and label-scarce segmentation, validate the superiority of our solutions over the SOTA methods.In summary, the contributions of this work are three-fold:• Revisit of a fundamental problem: To the best of our knowledge, we are the first to point out that the fundamental issue of feature selection remains unresolved in the realm of diffusion feature. • Generic insights: The properties we find are universal among different diffusion U-Nets, which can provide valuable insights for future work. • Extensive validation: Extensive experiments on three discriminative tasks validate the effectiveness of the feature selection solutions induced by our insights. 
related work: 
 Diffusion models for discrimination. We discuss three mainstream ways to use diffusion models for discrimination. (i) Diffusion classifier [26, 7] utilizes Bayes' theorem to transform a pre-trained diffusion model into an image classifier. This method enjoys a theoretical guarantee and does not need additional training. However, it is limited to image-level tasks. (ii) The second way is to model discriminative tasks as image-to-image generation tasks with diffusion models [22, 23] . This method is suitable for various dense vision tasks but requires heavy training. (iii) Diffusion feature [2, 58, 56, 41, 29] , the focus of this paper, follows the traditional practice of feature extraction to pursue the balance between wider applicability and less training. This makes it adaptable for different tasks and alleviates the training needs for the diffusion model. Only small downstream models may require training.The diffusion feature approach has seen various improvements. Some techniques toggle the input settings of diffusion models, such as seeking better timesteps [60, 29] . The others add trainable parameters outside the diffusion model to refine the outputs or provide an efficient fine-tuning alternative [58, 27, 46] . Additionally, some studies explore completely training-free methods that utilize spatial attention information [51, 43] , and some attempts focus on text-free diffusion models [32, 33] .Despite the progress, previous methods only consider activations between neighboring modules, leaving many potential candidates unevaluated. Our work shows that such an overlook can hinder model performance. By introducing a more comprehensive feature selection solution, our method could generically enhance both existing and future diffusion feature approaches.Analysis on model properties. The inner properties of neural networks have consistently received much attention [5, 34] . For transformers, Geva et al. [15] show that the feed-forward layers act as key-value memories and are interpretable for humans. Amir et al. [1] extend this insight to vision transformers and put it into practical applications such as image classification, while Vilas et, al. [45] try to make more detailed interpretation of ViT activations.The methodology of these studies provides valuable guidance for our research. However, since diffusion models are trained for generation, our study relies more on qualitative analysis and feature visualization compared to previous work. 
preliminaries: architecture of diffusion u-nets: 
 Diffusion models typically involve a forward pass and a reverse pass [21] . During the forward pass, noise is gradually added to a clean image x 0 ∈ R 3×w×h until the image resembles Gaussian noise, where w and h denote the width and height, respectively. This process can be denoted by x t ∼ q(x t |x 0 ), where q represents the noise posterior and t is the timestep. In the reverse pass, an end-to-end neural network ϵ θ , as parameterized by θ, learns to predict the noises and thus reconstruct the image. One such denoising step can be denoted as ϵ = ϵ θ (x t , t, c), where ϵ is the predicted noise, and c is the condition that describes the expected image content. Although there are alternative formulations for diffusion models [39, 28, 40] , they all rely on this neural network backbone ϵ θ . This study focuses on this backbone, typically implemented as U-Net [38] , rather than other components of diffusion models. Next, we will detail the architecture of the diffusion U-Net and standardize the terminology referring to different parts of the model, as shown in Figure 2 .We start with an overview. Following the initial U-Net design [38] , the U-Net has three main stages: down-stage, mid-stage, and up-stage. The down-stage reduces the resolution of activations, while the up-stage increases it. Both stages contain multiple resolutions, in each of which the activations share the same resolution. Furthermore, each resolution includes several modules, including ResModule (convolutional ResNet [20] structures), ViT Module [12] , and Downsampler/Upsampler (simple convolutional layers). Previous diffusion feature approaches only consider activations between these adjacent modules, i.e., inter-module activations.We next dive into the details below the module level. Among these modules, ResModule and ViTs adopt residual connection [20] , where an increment activation is added element-wise to the residual activation to refine it. Specifically, ResModule uses simple convolutional layers to produce increments, whereas ViTs use complex attention mechanisms, which will be further explained next. Typically, ViT operates as a standalone model followed by a decoder that produces the output predictions for visual tasks [12, 25, 9, 18] . However, in the diffusion U-Net, multiple ViTs are embedded into the network, and their outputs serve as increment activations.Furthermore, each ViT module consists of several stacked basic blocks. A basic block typically has a self-attention layer to perform attention on the image itself and a feed-forward layer , essentially a two-layer MLP [15] . Modern diffusion U-Net introduces an additional cross-attention layer between the two layers, enabling the fusion of the image and additional textual prompts. Besides, each layer includes a residual connection, meaning that the increment activation added to the outer residual is also the internal residual.In a nutshell, the architecture described above provides abundant activations that can serve as dense features. Given the massive activations, it is no longer operational to perform a full-scale quantitative comparison. Hence, we next make a comprehensive analysis of model properties to better understand these activations, which can help the qualitative filtering.Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof? Answer: [NA]• It should be clear whether the error bar is the standard deviation or the standard error of the mean. • It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified. • For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates). • If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text. 
distinct properties of diffusion u-nets: 
 The diffusion U-Net has many interesting properties, but now we only focus on those distinct from the knowledge of traditional U-Net [38] or ViTs [44] . As shown in Figure 3 , this section highlights three noticeable properties, each of which corresponds to a different level of the diffusion U-Net architecture described in Section 3. Notably, these properties can be universally observed in different samples and diffusion models, though all visualization in the main content is conducted on the same image and SDXL model for consistency. Additional visualization is provided in Appendix A. Besides, the omitted properties are available in Appendix E. The first property is at the macro level and closely related to the overall diffusion process. It is common that high-frequency signals are typically noisy [5, 34] . This phenomenon can also be observed in diffusion U-Nets, especially within the increment activations of residual connection. However, this does not mean that low-frequency signal activations in diffusion U-Nets are free from noises. As shown in Figure 3 (a), the diffusion process introduces a new type of noise that also impacts low-frequency signals. This is not surprising since the diffusion process requires the backbone to process noisy inputs and predict noises as outputs. As a result, activations near the inputs or outputs, regardless of their frequency, also suffer from such noises. Considering the special cause, we name such noises diffusion noises.How does the influence of diffusion noises spread across diffusion U-Nets? As shown in Figure 3 (a), diffusion noises exist throughout the entire down-stage, with a decreasing magnitude. Remarkably, during the early half of the up-stage, activations are rather clean, with no perceivable diffusion noises. Only in the later half do diffusion noises start to resurface. The existence of this asymmetric behavior can also be indirectly supported by the ablation curves in many fellow studies such as [2, 32, 33] . Furthermore, as shown in Appendix A, such an asymmetric pattern is consistent across activations in different diffusion models.Similar to common high-frequency noises, diffusion noises can degenerate feature quality. Hence, this property can serve as a criterion for identifying and filtering out sub-optimal activations, which we will delve into in Section 5.The second property is at the in-resolution level and closely related to recent advances in diffusion architectures. Specifically, the design of U-Net follows the idea of resolution hierarchy [38] . Consequently, the overall architecture displays a fine-coarse-fine granularity trend, looking like the alphabet "U". Traditional U-Nets implement this architecture with a relatively large number of resolutions, while each resolution is typically small, equipped with two or three simple convolutional layers.Hence, the understanding of traditional U-Net focuses on the granularity changes across resolutions, implicitly assuming that the change within a single resolution is negligible.However, diffusion U-Nets become much "fatter". In other words, modern diffusion U-Nets typically have much fewer resolutions, but each resolution is significantly enlarged. For example, SDv1.5 only has four resolutions [37] , and SDXL further decreases this number to three [36] , as shown in Figure 2 . Meanwhile, each resolution can produce much more activations, primarily thanks to the embedded ViT modules. This architectural evolution makes the granularity change within a single resolution more significant, as depicted in Figure 3(b) .Different granularity carries varied information and quality, resulting in different discriminative performance on downstream tasks [2, 29] . Hence, this discovery of in-resolution granularity changes highlights the necessity to evaluate more activations, especially those within the embedded ViTs, as they are of different granularity.For the third distinct property, we delve into the sub-module level, i.e., the blocks in embedded ViT modules. Positional embeddings, which are widely used in language transformers [44] and ViTs [12] , aim to provide spatial information for each input token. Consequently, the activations of traditional ViTs display strong positional information, where the latent pixel resembles nearby pixels more than those that are semantically similar but far away [1] . This phenomenon is significant for the layers close to the inputs. When the layer goes deeper, the tokens are refined with semantic information, making the activations display less positional information. However, only in the last few layers, such positional information becomes negligible.In contrast, ViT modules in diffusion U-Nets do not use positional embeddings [37, 36] , perhaps because the other U-Net components have provided sufficient spatial cues. This change results in distinct properties of the activations. On one hand, positional information is negligible for most activations despite how near they are to the inputs. For example, even in the first basic block, crossattention query activations contain no perceivable positional information. On the other hand, the queries and keys of self-attention still display non-negligible positional information, marked with orange circles in Figure 3 (c). Specifically, the latent pixel on the horse's neck is a light blue color, similar to the pixels to its left that actually represent the background. In contrast, the pixels above the circle are in purple color, though they also represent the horse's neck. Such comparison shows that a latent pixel is more similar to other pixels that are spatially near it than those semantically closer to it.We name this phenomenon locality since it has a different mechanism from that induced by positional embeddings. As pointed out in [12] , self-attention allows ViT to integrate global and local information even in the shallow layers, and the attention scope enlarges w.r.t. layer depth. Even without positional embeddings, self-attention activations are generally consistent with this insight, leading to the existence of locality. Nevertheless, the magnitude has indeed greatly reduced, compared to the visualization of conventional ViT activations in [1] . As shown in Figure 3 (c), in shallow activations, locality exists but is inherently weaker, as much semantic information is still reserved. In addition, locality quickly diminishes as the layer goes deeper.Since positional information can degrade the quality of activations [1] , its absence has the potential to enhance the activations in ViT modules. Moreover, locality can play a special role in activation filtering, as presented in Section 5.Although the visualization in Figure 3 is conducted only on SDXL, the scope of these properties is not limited to the specific architecture. Further supporting evidence is available in Appendix A.(i) Diffusion noises directly arise from the diffusion process. Hence, it is promising to extend this property to other diffusion backbones.(ii) In-resolution granularity changes come from the "fatness" of U-Nets, making it potentially applicable to more traditional U-Nets.(iii) Locality originates from the self-attention mechanism in ViT architectures, so it is broadly applicable to standalone or embedded ViTs where positional embeddings are absent.  
enhanced feature selection from diffusion u-nets: 
 So far, we have had a more comprehensive understating of the properties of diffusion U-Nets. All these properties, especially in-resolution granularity changes, encourage us to reconsider the feature selection solution, with a special emphasis on the activations in ViT modules. With these properties, we are also able to filter out many low-quality activations qualitatively, followed by a thus simplified quantitative comparison.Avoiding Diffusion Noises. As shown in Figure 4 (a), diffusion noises tend to degrade the quality of activations. Hence, it is natural to filter out the activations severely affected by diffusion noises from the candidate pool. Specifically, according to the asymmetric trend of diffusion noises, we only consider activations in the early half of the up-stage, which are rather clean. This approach will significantly reduce the number of candidate activations and simplify the quantitative comparison.Avoiding Self-Attention Locality. The locality in self-attention modules is another important factor that can degrade the activations. The empirical evidence in Figure 4 (b) demonstrates that these activations are generally inferior to the others, such as those from cross-attention layers or the outputs of ViT basic blocks. Consequently, it is rather safe to filter out most activations in self-attention modules from our candidate pool.Using Locality to Suppress Diffusion Noises. So far, the activations in the candidate pool are clean and free from locality. However, all these activations are low-resolution ones since highresolution activations are generally noisy and thus filtered out. This is unfavorable since some detailed information might only exist in high-resolution activations. To address this issue, we exploit a side effect of self-attention locality. Specifically, as indicated in Figure 4 (c), locality can help suppress diffusion noises via its focus on spatial structures. Although locality is sub-optimal, it is still superior to severe diffusion noises. In view of this, the candidate pool reserves self-attention activations extracted from the later half of the up-stage.We have filtered out many candidate activations based on the distinct properties of diffusion U-Nets. Additionally, all increment activations in residual connection can be further filtered out since they introduce high-frequency noises [5, 34] . After such qualitative filtering, a small but high-quality candidate pool is available. Taking SDXL as an example, the number of candidates decreases from 279 to 63, i.e., a 78% reduction. Next, we explain how to conduct this quantitative comparison briefly. The quantitative comparison follows the protocol of [2] . Specifically, given an input image x 0 , we first perform the forward pass with a pre-defined timestep t to generate the noisy image x t . Then, the U-Net backbone conducts one denoising step. Instead of collecting the model output ϵ, we gather U-Net activations and consolidate them to the candidate pool, as described in Section 5.1. Afterward, each activation is individually fed to a downstream model to evaluate its discriminative potential, with the details of the model described in Appendix C. Such comparison is made fair by using the same dataset for all activations. Finally, we can obtain the capability ranking of activations from each resolution, according to which features can be selected wisely. Notably, we conduct such comparisons on multiple datasets to guarantee generalizability, since the best activations may differ among different scenes [2] . Thanks to qualitative filtering, the time cost for each (model, dataset) pair has been reduced by more than one week, equipped with Nvidia(R) RTX 3090 GPUs.Given the capability ranking, feature selection is in fact flexible, as it is possible to combine multiple activations and specifically tune the choice for a task. For practicality, we provide off-the-shelf feature combinations for SDv1.5 and SDXL that are likely to generically perform well in Appendix B, according to the results detailed in Appendix D. Each of them consists of four activations mainly from ViT modules rather than inter-module positions. Taking SDv1.5 as an example, one of the four selected features is from one self-attention layer in the highest resolution, which utilizes our observation in Section 4. 
experimental validation on multiple discriminative tasks: 
 To validate the effectiveness of our feature selection solution, the experiments are conducted on three popular discriminative tasks: semantic correspondence, semantic segmentation, and label-scarce segmentation. The SOTA methods for each task are selected as competitors. Besides, we also compare with two baselines that select the conventional inter-module activations as features [2] , named Legacy-v1.5 and Legacy-XL. For our method, we provide the following implementations:• Ours-v1.5 & Ours-XL: Features extracted from SDv1.5 and SDXL, respectively.• Ours-XL-t: For fairness, we further enhance SDXL features with some additional techniques that are also adopted by SOTAs. The techniques we select, i.e., attention maps [58] and feature amalgamation [2, 29, 60] , are relatively simple and lightweight.More experimental details are in Appendix C, such as task information, evaluation metrics, and implementation details. Besides, Appendix D presents additional results not covered here.We present the experimental results for semantic correspondence in Table 1 . From the results, we have the following observations: Previous approaches either reserve the noisy activations and thus suffer from performance degradation [2, 52] , or simply discard high-resolution activations and thus suffer from information loss [58, 46] . In contrast, our approach uses self-attention locality to suppress diffusion noises and harvest better high-resolution activations. (ii) Legacy-XL is inferior to Legacy-v1.5 (66.00 v.s. 75.14 on PCK@0.1 img ). At first glance, this result is counter-intuitive since SDXL is more advanced than SDv1.5. However, the analysis in Section 4 can unravel the mystery. Specifically, since SDXL has more ViT modules, the more valuable activations shift from inter-module positions to these embedded ViTs. Since the baseline does not consider ViT modules, Legacy-XL fails to achieve better performance.In contrast, Ours-XL shows improvement over Ours-v1.5 (81.72 v.s. 77.78 on PCK@0.1 img ). This is consistent with the advance in model architecture and again validates our analysis. (iii) Ours-XL-t significantly outperforms the SOTA method with a similar amalgamation technique, i.e., DHF [29] (83.90 v.s. 72.56 on PCK@0.1 img ). This performance gain again validates the effectiveness of our method.As shown in the left part of Table 2 , Ours-XL-t and Ours-XL achieve state-of-the-art performance on the semantic segmentation task (45.71 and 43.45 v.s. 40.89 on ADE20K), demonstrating its effectiveness and generalizability. Furthermore, the most competitive SOTA, Meta Prompts [46] , introduces a large number of trainable parameters and uses the diffusion U-Net recurrently, which is rather time-consuming. In contrast, our method delivers superior results with efficiency maintained.Unlike the results in semantic correspondence, Legacy-XL outperforms Legacy-v1.5 on CityScapes (71.67 v.s. 64.01), and the performance gap between Legacy-v1.5 and Ours-v1.5 is narrow (64.01 v.s. 64.10). This is because this task utilizes a relatively large-scale downstream model, which can significantly refine the input features and thus reduce the gap in feature quality. Nevertheless, Ours-XL still achieves a significant improvement over Legacy-XL (74.47 v.s. 71.67).One advantage of diffusion features is the applicability to label-scarce scenarios [2] . For validation under such conditions, we experiment on the label-scarce segmentation task, with results presented in the right part of Table 2 . The observations are generally similar to those on the semantic correspondence task. For example, Ours-v1. Next, we focus on the comparison with the SOTA method, DDPM [2] . Although DDPM is a relatively early study, it outperforms the other competitors and our most implementations. This result has two reasons. On one hand, DDPM performs the diffusion process directly in the image space rather than the currently common practice of compressed latent space. Although inefficient, such an implementation yields better discriminative features. On the other hand, DDPM utilizes diffusion models specifically trained on each dataset. In comparison, we use pre-trained general-purposed SDv1.5 or SDXL to be more efficient and thus consistent with the motivation of diffusion feature. Hence, it is rather challenging to surpass this SOTA method. Fortunately, our best implementation, Ours-XL-t, achieves this goal with the help of additional lightweight techniques (66.3 v.s. 65.0). 
conclusion and future work: 
 In this study, we revisit the fundamental problem of feature selection from diffusion U-Nets. We point out that prior arts only consider a limited range of potential activations. In contrast, we consider a much wider range of activations as candidates, especially those extracted from the embedded ViT modules. Given the large volume of the candidate pool, we first analyze the properties of diffusion U-Nets. The properties we find are universal such that our observations are not limited to the specific diffusion architecture. Based on these properties, we qualitatively filter out many activations with low quality, facilitating the following quantitative comparison. On top of this, concrete feature selection solutions are proposed for two popular diffusion models, i.e., SDv1.5 and SDXL. Finally, extensive experiments on three discriminative tasks validate the effectiveness of our method.However, we are not sure whether our observations can generalize well to recently-developed DiT models [35] since they have a markedly different architecture from U-Net-based diffusion models. Thus, analyzing DiT models is a promising topic for future research. 
experiments compute resources: 
 Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?Answer: [Yes] Justification: We provide information in Appendix G.Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage. • The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute. • The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper). 
code of ethics: 
 Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines? 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed? 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Justification: The paper poses no such risks.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes] Justification: We cite the original papers where related assets are first mentioned and list their license in Appendix H.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. 
 
 Appendix: A. We have provided activation visualization from SDXL in a simple outdoor scene presenting a horse in the wild. Next, we will show visualization from the same model in different scenes to demonstrate the universality of the observed properties. Another simple outdoor scene presenting a cat is in Figure 5 .Two simple indoor scenes are visualized in Figure 6 and Figure 7 . Two complex outdoor scenes of urban streets are visualized in Figure 8 and Figure 9 . Two complex indoor scenes are shown in Figure 10 and Figure 11 . Playground v2 Activations. Playground v22 shares the same architecture as SDXL but is trained independently, and it is claimed to be more powerful in generation. In Figure 12 , we present its activation visualization using the same horse image as the primary SDXL visualization. Compared to SDXL, Playground v2 activations are less noisy, particularly in the down-stage. This supports the claim that Playground v2 is a stronger model.SDv1.5 Activations. SDv1.5, as an older model, has a slightly different architecture from SDXL. Specifically, SDv1.5 has four resolutions instead of three, but its ViTs contain only one layer. Moreover, in SDXL, only the highest resolution lacks ViT due to efficiency concerns, while in Figure 9 : Visualization of SDXL activations on a complex outdoor scene.Figure 10 : Visualization of SDXL activations on a complex indoor scene.SDv1.5, only the lowest resolution lacks ViT due to its low resolution. Despite the architectural changes, the three unique properties still apply to SDv1.5, as shown in Figure 13 .Video Diffusion Activations. To further demonstrate the universality, we even select a diffusion model for video generation [6] for visualization. This model is based on the SDv1.5 architecture, with an additional temporal attention layer inserted after cross-attention to enable sequential generation. Although this model is designed for a different task, we can still observe the three unique properties in Figure 14 .Conventional U-Net Activations. As comparison to diffusion U-Net, we also visualize some activations from a conventional U-Net in Figure 15 , where the three properties of diffusion U-Net can hardly be observed. In this section, we index all diffusion U-Net components in the order in which they are activated during a network forward run. Besides descriptions in natural language, we also index the activations using the same notations as used in our code implementation. These notations denote stages using down/mid/up, resolutions as level, and module index as repeat. Index starts to count from 0 instead of 1, following the convention in coding.B.1 Feature Selection Solution for SDv1.5We select four activations from SDv1.5, maintaining the same total feature channels as the conventional way to extract all output activations from each resolution.(i) The cross-attention query activation from the 2nd ViT, the 2nd resolution. This provides coarse information for simple scenes (up-level1-repeat1-vit-block0-cross-q).(ii) The inter-module activation after the 3rd ResModule, the 2nd resolution. This provides coarse information for complex scenes (up-level1-repeat2-res-out).(iii) The cross-attention query activation from the 2nd ViT, the 3rd resolution. This provides finer information with a higher resolution (up-level2-repeat1-vit-block0-cross-q).(iv) The self-attention key activation from the 1st ViT, the 4th resolution. This extracts features from the highest resolution, harnessing the noise suppression effect of self-attention locality (up-level3-repeat0-vit-block0-self-k).We omit the index of basic blocks in ViTs, as SDv1.5 only contains one-layer ViTs. Additionally, we totally ignore the lowest resolution, as its activations have very low resolution (8 × 8 if the input image is 512 × 512), suggesting inferiority, as supported by the quantitative comparison.Four activations are selected from SDXL, trying to get similar total feature channels to the SDv1.5 feature selection solution.(i) The output activation after the 8th basic block, the 1st ViT, the 1st resolution. This provides relatively coarse information for simple scenes (up-level0-repeat0-vit-block7-out).(ii) The output activation after the 6th basic block, the 1st ViT, the 1st resolution. This provides relatively coarse information for complex scenes (up-level0-repeat0-vit-block5-out).(iii) The cross-attention query activation from the 1st basic block, the 1st ViT, the 2nd resolution. This provides relatively fine information for simple scenes (up-level1-repeat0-vit-block0-crossq).(iv) The output activation after the 1st basic block, the 1st ViT, the 2nd resolution. This provides relatively fine information for complex scenes (up-level1-repeat0-vit-block0-out).We ignore the highest resolution since it is affected by diffusion noises and lacks ViTs from which we can extract self-attention activations. However, if the downstream model is strong enough to learn to suppress noises, it may be possible to additionally extract inter-module activations from this resolution to harness more information.For the setting Ours-XL-t, we mainly utilize two simple techniques that are also adopted in some SOTA methods: (i) We additionally extract attention maps, i.e., the similarity scores of cross-attention query and key, as dense features [58, 52] . Such attention maps are closely related to the semantics of prompts, thus providing important supplementary information. Despite its usefulness, this technique only adds a few additional channels to the features. (ii) We amalgamate features from different models [2, 29, 60] through simple concatenation. This technique is a common practice and can be seen as an extension of amalgamating different activations together as a whole feature. We next explain what activations are selected to implement the two techniquess.We first extract features according to the feature selection solution for SDXL. Afterward, we extract additional features from SDv1.5:(i) The cross-attention query activation from the 2nd ViT, the 2nd resolution (up-level1-repeat1vit-block0-cross-q).(ii) The cross-attention query activation from the 2nd ViT, the 3rd resolution (up-level2-repeat1vit-block0-cross-q).(iii) The upsampler output activation from the 3rd resolution, to harness high-resolution information (up-level2-upsampler-out).(iv) The self-attention key activation from the 1st ViT, the 4th resolution, to harness high-resolution information (up-level3-repeat0-vit-block0-self-k).(v) The attention maps averaged over all cross-attention layers in the up-stage.We also extract one feature from Playground v2: the output activation after the 4th basic block, the 1st ViT, the 1st resolution (up-level0-repeat0-vit-block3-out).Large-scale datasets for semantic segmentation mostly consist of images of complex scenes. In such cases, we find attention maps can be too noisy to be useful. Therefore, we discard the attention map technique and the entire SDv1.5 model, as it is weaker compared to the newer SDXL and Playground v2 models. To compensate for the loss of activations, we select additional activations from SDXL and Playground v2.From SDXL, we select all the activations as described in the feature selection solution for SDXL and select one additional activation: the upsampler output activation from the 2nd resolution, to harness high-resolution information (up-level1-upsampler-out). From Playground v2, we select the following activations:(i) The output activation after the 6th basic block, the 1st ViT, the 1st resolution (up-level0-repeat0vit-block5-out).(ii) The cross-attention query activation from the 1st basic block, the 1st ViT, the 2nd resolution (up-level1-repeat0-vit-block0-cross-q).(iii) The upsampler output activation from the 2nd resolution (up-level1-upsampler-out).Task and Dataset. Semantic correspondence [18] involves finding a pixel in an image that semantically matches another keypoint pixel in a reference image, such as the hind legs of two different cats.We conduct experiments on the SPair-71k dataset [30] .Evaluation Metric. PCK@0.1 img (↑) and PCK@0.1 bbox (↑) are used, following the widely-adopted protocol reported in [30] . These two metrics mean the percentage of correctly predicted keypoints, where a predicted keypoint is considered to be correct if it lies within the neighborhood of the corresponding annotation with a radius of 0.1 × max(h, w). For PCK@0.1 img /PCK@0.1 bbox , h, w denote the dimension of the entire image/object bounding box, respectively.SOTA Competitors. We provide the results from four SOTA methods: DINO [4] and DHPF [31] as non-diffusion-feature methods, as well as DIFT [41] and DHF [29] as diffusion feature methods.Implementation Details. The semantic correspondence task can be done via the nearest neighbor algorithm [42] , which is unsupervised and training-free [30] . We add one additional trainable convolutional layer before applying the nearest neighbor algorithm to refine the input features, which is also adopted by some SOTAs including DHF. The model is trained for two epochs, each containing 5,000 sample pairs, following conventional settings. Our implementation is derived from DHF, and we keep all hyper-parameters at their default settings.Task and Dataset. Semantic segmentation [18] is essentially pixel-level classification. For this task, we choose the ADE20K dataset [59] with over 20k annotated images of 150 semantic categories, and the CityScapes dataset [8] , which contains 5,000 fine annotated images of urban street scenes.Evaluation Metric. We use mIoU metric, which is the mean over the IoU performance across all semantic classes [18] . For each image, IoU (Intersection over Union, ↑) is defined by #(overlapped pixels between the prediction and the ground truth) / #(union pixels of them).SOTA Competitors. We choose three diffusion feature SOTA methods as competitors. ODISE [52] is an early method with a simple implementation. VPD [58] is another early study of this field, which introduces additional text adapter modules for improvement. Meta Prompts [46] is a newer method and shows significant improvements. We also report the performance of MaskCLIP [11] , which is included as a competitor in the ODISE study.Implementation Details. Both VPD and Meta Prompts perform full-scale fine-tuning on the diffusion U-Net using feedback from the discriminative task. This heavy fine-tuning does not entirely comply with the motivation of diffusion feature, which seeks a balance between wider applicability and less training, and is hard to extend to the larger SDXL model. Therefore, we keep the entire diffusion model frozen instead. As the setting has been changed, the performance of VPD and Meta Prompts in Table 2 is based on our experiments, not the reported results from the original papers. Our implementation directly uses the hyper-parameters reported in Meta Prompts.Task and Dataset. Using features from a pre-trained diffusion model ensures good performance even when labeled training data is scarce [2] . For this setting, we use a dataset collected in [2] and experiment on its Horse-21 subset, the data of which is sourced from LSUN [54] . This subset contains only 30 labeled training images to be consistent with the intuition. The semantic segmentation in the label-scarce scenario also uses the mIoU metric.SOTA Competitors. We select the SOTA diffusion feature approach, DDPM [2] , as the major competitor. We also include other representative segmentation methods: DatasetDDPM, MAE [19] , SwAV [3] , which are all reported in [2] .Implementation Details. Following DDPM [2] , the downstream model is an ensemble of ten simple MLP networks, each conducting pixel-wise classification. The simplicity of the model is intended to demonstrate the innate capability and generalizability of diffusion models. Our implementation is derived from DDPM with only batch size changed among all hyper-parameters. We use a larger batch size for faster experiments as a smaller one does not improve performance. Additionally, this is also the setting for the quantitative comparison, as the compact size of the dataset can enhance efficiency.As stated in Section 5, it is preferable to conduct the quantitative comparison across multiple datasets and choose activations that are optimal for each. This approach can enhance the generalizability of the selected features. To evaluate the generalizability of our features, we conducted an additional experiment, with results presented in Table 3 .In this experiment, we design an alternative feature selection solution for SDXL, based solely on quantitative results from a single dataset consisting of simple scenes. In this solution, we extract both optimal and slightly sub-optimal activations, maintaining the same total number of feature channels as the standard solution. The alternative solution achieves higher performance on the simple scene it is based on but performs significantly worse on the other scene. Therefore, we conclude that our standard feature selection solution achieves generalizability across different scenes, albeit with a slight performance drop compared to features specifically selected for each scene. In this part, we present all the quantitative comparison results obtained following the protocol described in Section 5. These results are from a dataset of simple scenes (Horse-21 [2] ) and a dataset of complex scenes (Bedroom-28 [2] ). Since SDXL and Playground v2 share the same U-Net architecture, their results are shown in the same tables. We display the results from the lowest resolution in Table 4 and the results from the middle resolution in Table 5 . We have also done a quantitative comparison on SDv1.5, and the results are shown in Table 6 . Typically, high-frequency signals are usually noisy [5, 34] . This applies to the diffusion U-Net, and we can further examine what activations are more vulnerable to such noises. To be specific, the diffusion U-Net contains many residual connection structures, and their increment activations are high-frequency signals prone to noises. Such increment activations include:(i) The increment branch of ResModule. These activations are moderately noisy and thus usually less effective than inter-module activations. (ii) The feed-forward layer activations within ViTs. These activations are also moderately noisy.However, this results from their significantly more channels compared to other activations, which can reduce their noise magnitude. (iii) The self-attention value activations within ViTs. These activations are severely noisy and suffer significant degradation as they are the nested inner increments within embedded ViTs.These activations are visualized in Figure 16 for a clearer illustration. Additionally, the residual activations within ViTs are at the same time also increments to the main inter-module residual. However, these activations are not obviously affected by high-frequency noise, possibly due to their dual role as ViT residuals.We have previously described the existence of in-resolution granularity changes, and this section will further detail the pattern of one such change. In one resolution of the up-stage, activations gradually shift from coarse to fine granularity, which aligns with intuition. However, the diffusion U-Net tends to overly refine the inter-module activations, resulting in slight noises in the last few inter-module activations due to excessive detail. This can be observed from the visualization of Figure 17 . Consequently, a drop in discriminative performance is often seen near the end of one resolution.Multiple ViTs exist within one resolution; for example, one resolution in the up-stage typically contains three ViTs. These ViTs collaborate in refining the main inter-module residual. During this process, each ViT exhibits an inner granularity change as it produces the increment activation, and the changes of collaborating ViTs form a certain pattern. To be specific, the change of one ViT overlaps with the previous one to some extent but also shifts as a whole towards finer granularity, which is shown in Figure 18 .It might be a good future direction to focus on more challenging discrimination scenarios. Specifically, long tail and out-of-distribution are important problems in discrimination, which can greatly hinder the performance of models that work well under i.i.d. settings [49] . There have been a few attempts to address this more challenging problem with diffusion models. For example, the disentanglement property of prompts might grant diffusion features cross-domain capability [16] . It is also possible to utilize diffusion models to synthesize training samples to adjust training data distribution [50, 55] . Given these attempts, more efforts are still required to be put in this direction. Moreover, there is a recent study [17] that might boost long tail and out-of-distribution discrimination using diffusion models. To be specific, AUC is an evaluation metric as well as a loss function that promotes good performance on both head and tail samples, which is an important tool for long tail and out-ofdistribution studies [47, 48] . Conventionally, AUC is applicable to image classification but not semantic segmentation and other pixel-level tasks, but the aforementioned study manages to adopt AUC for semantic segmentation. Given this new tool, it is now made more viable to attempt to enhance diffusion feature on long tail and out-of-distribution problems.We use Nvidia(R) RTX 3090 and Nvidia(R) RTX 4090 GPUs for the experiments, all with 24GB VRAM. Most of our experiments, except label-scarce segmentation, require no additional storage besides the necessary space for model checkpoints and datasets. The label-scarce segmentation task first extracts features and stores them on the disk, and then loads them for the downstream task, which takes about 4GB.The codes are designed to be able to run on a single GPU or less than 4 GPUs, while multiple experiments can run simultaneously if more GPUs are provided. Each experiment on the semantic correspondence task takes about 5 hours. Each experiment on the large-scale semantic segmentation task takes 2 to 3 days, depending on the dataset. Each experiment on the label-scarce segmentation task takes about 1 hour, but we repeat on 5 random splits following [2] , which increases the overall time. All the experiments in sum can be done within two weeks. Justification: We discuss the limitations in Section 7, which are mainly related to DiT models [35] . Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.Justification: We conform with the NeurIPS Code of Ethics.Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).Justification: There is no socetal impact.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.• The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).https://huggingface.co/playgroundai/playground-v2-1024px-aesthetic 


Decision: The paper considers the problem of feature selection within diffusion models, analyzing various properties of diffusion models (properties such as asymmetric diffusion noises, in-resolution granularity changes, and locality without positional embeddings), and proposing a feature selection method that outperforms state-of-the-art techniques across multiple tasks.  Empirical results are
detailed and validate the claims.  All reviewers agree with the4 value of the contribution.

Reviews: review: paper_summary: paper_summary: This paper proposes a new feature selection method for diffusion models by evaluating a broader range of activations, particularly those in embedded Vision Transformers (ViTs). The authors identify the limitations of current approaches that consider only a narrow range of activations and introduce a qualitative analysis to filter out low-quality activations in diffusion U-Nets. They develop specific feature selection solutions for popular diffusion models SDv1.5 and SDXL. Experiments demonstrate that this method outperforms state-of-the-art techniques in tasks like semantic correspondence, semantic segmentation, and label-scarce segmentation, showcasing its effectiveness and generalizability.
strength_weakness: strength_weakness: - As pointed out by the authors, the observations and methods proposed do not necessarily generalize well to recently developed DiT models since their architecture is markedly different from diffusion U-Nets .
- Some details can be more clarified. For example, in Appenidx C.3, the authors state that “this is also the setting for the quantitative comparison”. In other words, quantitative comparison is conducted on Label-Scarce Segmentation, and the experiments on the other tasks follow this setting. Is this understanding correct? If so, I wonder why the authors select this setting to conduct quantitative comparison. 
- Besides, there are some typos. For example, in the introduction, “a fundamental problem to select” should be “a fundamental problem in selecting”. In the caption of Figure 3, “existing knowledge of other models” should be “existing knowledge about other models”.
questions: questions: Please refer to the weakness.
limitations: limitations: Yes, the authors have clarified the limitations in Section 7.
paper_summary: paper_summary: Diffusion models have achieved significant success in image generation and show great potential for various discriminative tasks. The authors rethink the foundational problem of feature selection within these models. To this end, they analyze the properties of diffusion models, including asymmetric diffusion noises, in-resolution granularity changes, and locality without positional embeddings, to filter out suboptimal activations. The results demonstrate that their proposed feature selection method outperforms state-of-the-art techniques across multiple discriminative tasks, achieving superior performances.
strength_weakness: strength_weakness: -  The authors acknowledge the uncertainty about whether the findings can generalize to newer models like DiT (Diffusion Transformer) due to architectural differences.
- The qualitative filtering approach proposed is novel, but its scalability and efficiency in very large-scale settings are not fully demonstrated. How do the authors alleviate this problem?
- The paper focuses on empirical results and qualitative analysis. There is a lack of rigorous theoretical foundation or mathematical proofs to support the proposed feature selection methodology and the identified properties of diffusion U-Nets. A more robust theoretical framework would strengthen the claims made.
questions: questions: Besides those in weakness, I wonder can we combine multiple features during the qualitative analysis to further improve the performance? And how many features do you select for each image?
limitations: limitations: Yes, as stated in Sec.7.
paper_summary: paper_summary: This paper highlights the importance of considering a broader range of activations within diffusion models. The authors propose three universal properties of diffusion U-Nets that aid in qualitatively filtering out activations that are clearly sub-optimal. On top of this, the authors can improve the efficiency of feature selection. By leveraging these properties, the proposed method demonstrates superior performance across multiple discriminative tasks, such as semantic correspondence and segmentations. This comprehensive approach to activation selection addresses a fundamental issue in diffusion models, enhancing their applicability and performance in various tasks.
strength_weakness: strength_weakness: W1: This paper could benefit from introducing more notations to clearly define and differentiate between various activations and components within the diffusion models, which would further enhance the clarity of the descriptions.
W2: While the empirical results are robust, the paper could include more theoretical analysis to provide deeper insights.
W3: Some more latest related work are commended to be cited:
[1] Diffusion 3D Features (Diff3F): Decorating Untextured Shapes with Distilled Semantic Features, CVPR 2024. [2] DiffSeg: Towards Detecting Diffusion-Based Inpainting Attacks Using Multi-Feature Segmentation, CVPR 2024.
questions: questions: I have no more questions.
limitations: limitations: Yes
paper_summary: paper_summary: The paper addresses the critical task of enhancing feature selection in diffusion models to improve performance in discriminative tasks such as semantic correspondence, semantic segmentation, and label-scarce segmentation. Previous methods often overlooked many potential activations within diffusion models, leading to suboptimal performance and limitations due to ignoring high-resolution activations and not effectively handling diffusion noises. The authors propose a comprehensive feature selection solution that leverages distinctive properties of diffusion U-Nets, including diffusion noises, in-resolution granularity changes, and locality without positional embeddings, to filter and select the most relevant features. Experimental results demonstrate that their method outperforms state-of-the-art techniques, achieving significant improvements in various metrics across multiple datasets.
strength_weakness: strength_weakness: - Some more comparisons are recommended. While the authors claim to find unique properties in diffusion U-Nets, it lacks a detailed comparison with traditional U-Nets. A more comprehensive analysis of how these properties differ would provide better context.
- Some visualizations, such as Figure 3(c) showing positional information in self-attention activations, could be clearer and better explained to enhance understanding of the findings.
questions: questions: See the weakness.
limitations: limitations: Yes