SSLA: A Generalized Attribution Method for Interpreting Self-Supervised Learning without Downstream Task Dependency

Authors: Zhiyu Zhu, Jiayu Zhang, NAN YANG, Xinyi Zhang, Zhibo Jin, Jianlong Zhou, Fang Chen
Primary area: interpretability and explainable AI
Keywords: Interpretability, Attribution, Self-Supervised Learning

Abstract:
Self-Supervised Learning (SSL) is a crucial component of unsupervised tasks, enabling the learning of general feature representations without the need for labeled categories. However, our understanding of SSL tasks remains limited, and it is still unclear how SSL models extract key features from raw data. Existing interpretability methods are heavily reliant on downstream tasks, requiring information from these tasks to explain SSL models. This reliance blurs the line between interpreting the SSL model itself and the downstream task model. Moreover, these methods often require additional samples beyond the target of interpretation, introducing extra information that complicates the interpretability process.
In this paper, we propose three fundamental prerequisites for the interpretability of SSL tasks and design the Self-Supervised Learning Attribution (SSLA) algorithm that adheres to these prerequisites. SSLA redefines the interpretability objective by introducing a feature similarity measure, reducing the impact of randomness inherent in SSL algorithms, and achieving more stable interpretability results. Additionally, SSLA abstracts the interpretability process, making it independent of specific neural network architectures. To the best of our knowledge, SSLA is the first SSL interpretability method that does not rely on downstream tasks. We also redesign a more reasonable evaluation framework and establish baselines for comparative assessment. The source code for our implementation is publicly available at https://anonymous.4open.science/r/SSLA-EF85.

Decision: reject
Reviewer scores: 5, 5, 3, 5

Reviews and discussion:
<thread>
creation date:
1729847643126

thread_type:
official_review

<thread_object>
creator: reviewer_oq2z

<content>
summary:
This paper indicates that the introduced additional samples from downstream task would impede the interpretability of Self-Supervised Learning (SSL). To tackle this issue, the authors try to propose a new interpretability objective by introducing a feature similarity measure, decoupling the interpretability process from the reliance of downstream tasks.

strengths:
- The additional information introduced by prediction head and downstream tasks may influence the interpretability of SSL is a crucial and challenging  breakthrough point.
-  Three fundamental prerequisites for the interpretability of SSL proposed by this paper sounds reasonable.

weaknesses:
- Despite the author indicating the potential weakness for the interpretability of SSL, the alignment ability for data augmentation is just one side of current self-supervised learning. The other component, which is often regarded as an extra design to prevent training collapse intuitively, is essentially to ensure that the representation divergence is sharp enough to cluster the data distribution by latent categories. More details can be found in [1] and [2]. Therefore, only adopting the extent of augmentation that is invariant to evaluate the related influence of variables is quite biased.

   - [1] Awasthi, Pranjal et al. “Do More Negative Samples Necessarily Hurt in Contrastive Learning?” International Conference on Machine Learning (2022).
   - [2] Weiran Huang, Mingyang Yi, Xuyang Zhao, and Zihao Jiang. Huang, Weiran, et al. "Towards the generalization of contrastive self-supervised learning." arXiv preprint arXiv:2111.00743 (2021).

questions:
**Reviewing summary**
- As listed in the weaknesses, I think the authors did not incorporate the divergence into their consideration, which can be regarded as the most critical component of contrastive self-supervised learning, making their criterion sound unreasonable. Despite their indicating the unreasonable aspects regarding the interpretability of SSL in relation to downstream tasks, this results in my score of 3.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730516340297

thread_type:
official_review

<thread_object>
creator: reviewer_czec

<content>
summary:
The paper proposes SSLA, a feature attribution method for self-supervised learning (SSL) tasks. In particular, the method is designed without dependency of downstream tasks. The method starts by defining the usefulness of SSL model as its ability to preserve representation of data after transformation. Then it addresses the significance of features by attributing this usefulness to features iteratively. The paper then conducts feature masking experiment to demonstrate the effectiveness of the method.

strengths:
1. The paper proposed a novel feature attribution method for SSL model that does not rely on downstream tasks. 
2. The claim made in the paper is supported by both theoretical derivations and experiments. 
3. The paper is well written and easy to follow. The discussion of prerequisites of an attribution method for SSL may spark interesting discussions.

weaknesses:
1. The paper relies on the independence of downstream tasks, which make the comparison of this method and existing methods difficult. Hence, it is difficult to address the effectiveness of this method. 
2. The derivation of theorems rely heavily on first order approximation. Though it is common, the paper does not provide analysis on error bounds, which downgrades the trustworthiness of the method.
3. The two main components of the method lack motivation. The first one is using cosine-similarity of features before/after transformation as a measure of usefulness of SSL model. The correlation (or even causality) of this and "SSL as learning representation" is not clear. The second one is the iterative method. The author may consider justify why we need an iterative method to attribute the importance.
4. Although the paper proposes the method to be independent of downstream tasks, its evaluations still rely on downstream tasks, which seems counter-intuitive(Line 179 -180). Moreover, since the evaluation is dependent of downstream tasks, the author may consider compare their method to other SSL attribution methods that rely on downstream tasks.

questions:
See weaknesses.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730670750042

thread_type:
official_review

<thread_object>
creator: reviewer_bs4q

<content>
summary:
This paper addresses the interpretability of SSL models, focusing on the challenge that existing interpretability methods often rely on downstream tasks or specific model architectures. To overcome these issues, the authors propose three fundamental prerequisites for SSL interpretability:
1. The interpretation should not introduce information from downstream tasks.
2. The interpretation process should not introduce samples other than the current sample.
3. The interpretation process should not be restricted to specific model architectures.

Based on these prerequisites, they introduce the Self-Supervised Learning Attribution (SSLA) algorithm. SSLA redefines the interpretability objective by introducing a feature similarity measure. 
They also propose a new evaluation framework tailored to SSL tasks, arguing that traditional interpretability evaluation methods are impractical due to the absence of explicit labels and suitable baselines in SSL settings. Experiments are conducted using five representative SSL methods (BYOL, SimCLR, SimSiam, MoCo-v3, MAE) on the ImageNet dataset with ResNet-50 as the backbone. They compare SSLA against a random masking baseline, demonstrating that SSLA can more effectively identify important features that influence the SSL model's representations.

strengths:
* **Novel Focus on SSL Interpretability:** The paper addresses an important and under-explored area—the interpretability of SSL models without reliance on downstream tasks or specific architectures.
* **Clear Prerequisites:** The authors clearly outline three prerequisites for SSL interpretability methods, providing a solid foundation for their approach.
* **Architecture-Agnostic Approach:** SSLA is designed to be independent of specific neural network architectures, potentially making it broadly applicable across different SSL models.
Recognition of Evaluation Challenges: The authors recognize the limitations of traditional interpretability evaluation methods in the context of SSL and attempt to propose a new framework tailored to SSL tasks.

weaknesses:
* **Insufficient Empirical Evaluation:** The experimental evaluation is limited. The authors only compare SSLA to a random masking baseline. There are no comparisons with other existing attribution methods adapted to SSL, making it hard to gauge the effectiveness of SSLA.
* **Limited Dataset and Model Diversity:** Although experiments are conducted on the ImageNet dataset using ResNet-50, the evaluation lacks diversity in both datasets and model architectures. The claim that SSLA is architecture-agnostic is not fully supported without experiments on different architectures.
* **Evaluation Methodology Concerns:** The proposed evaluation framework is novel but not thoroughly validated. The authors argue that traditional evaluation methods are unsuitable for SSL interpretability but do not provide sufficient empirical evidence or theoretical justification. It is unclear whether the metrics used effectively measure interpretability in SSL contexts.

questions:
*  **Comparison with Existing Methods:** Have you considered adapting existing attribution methods like Integrated Gradients or Grad-CAM to SSL settings for comparison? How does SSLA perform relative to these methods?
*  **Validation of Evaluation Framework:** How have you validated the effectiveness of your proposed evaluation framework? Have you conducted any studies or experiments to show that it correlates with human intuition or ground truth attributions?
*  **Testing on Diverse Architectures:** Given that experiments are only conducted with ResNet-50, have you tested SSLA on other architectures?
*  **Handling SSL Randomness:** How does SSLA account for the randomness inherent in SSL methods, such as stochastic data augmentations? Does this randomness affect the stability of the attribution results?
*  **Computational Overhead:** What is the computational cost of SSLA compared to standard inference? Is it feasible to apply SSLA to large-scale models and datasets?
*  **Generalization to Other Domains:** Can SSLA be applied to SSL models in domains other than computer vision?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730686177408

thread_type:
official_review

<thread_object>
creator: reviewer_6yuv

<content>
summary:
The paper proposed a novel attribution algorithm for feature-level attribution on self-supervised learning. Compared to other feature-level attribution. The method is designed to meet prerequisites that the interpretation should not rely on 1) downstream task 2) other samples (other than the argumentation) and 3) model architectures. Authors present some experiments to justify the new method (SSLA) is effective.

strengths:
- The prerequisites are designed to resolve the problem caused by other factors. And the method are design to reflect this spirit.
- The diagram are clear and help the readers to understand the method.

weaknesses:
- I am not sure if the perquisites can be widely accepted by the community. For example, what is the downside (empirically / theoretically) if a downstream task is considered during the attribution process?
- Lack of comparison between different attribution methods. One interesting problem could be what's the difference between SSLA result v.s. other methods that relies on downstream tasks.
- Minor presentation suggestion
  - Equation 1 and 2 seems to be a little redundant.
- I am willing to raise the rate if the effectiveness of SSLA on some traditional evaluation methods (on downstream tasks) are proved (at least) to be correlated

questions:
- In Figure 2, why we have a full R^n shape for a0, a1, ..., a_T?
- There seems to be no reason seperate the snowflake and light blue arrow in Figure 2?
</content>
</thread_object>

</thread>


Metareview:
metareview:
This paper introduces a novel interpretability framework for SSL models, aiming to decouple attribution from downstream tasks. The method, SSLA, uses a feature similarity measure to explain SSL models independently of downstream tasks or additional sample information. The paper outlines theoretical derivations, empirical results, and a new evaluation framework.

While the idea is innovative, several key concerns remain unresolved. Reviewers criticized the lack of baseline comparisons with existing attribution methods, even if those rely on downstream tasks. The reliance on cosine similarity as the sole evaluation metric was seen as insufficient, with Reviewer oq2z emphasizing the importance of divergence-based measures, which were not incorporated. Reviewer CZeC pointed out the reliance on first-order approximations without error bounds, weakening the theoretical robustness. Empirical validation was also limited, with experiments primarily conducted on ResNet-50 and ImageNet, raising questions about generalizability.

Although the authors actively engaged in the rebuttal phase, they did not convincingly address these concerns. I also agree with parts of the reviewers' concerns and think that the proposed method does not yet meet the bar for acceptance. I thus recommend rejecting the paper, but I encourage the authors to incorporate these helpful discussions into the revision and resubmit to a future venue.