Paper: DIFAIR: Towards learning differenciated and interpretable representations (ICLR-2024)

Reviewer comment (Reproducibility):
It is not clear to me how you applied Maximum Output Score (MOS) on DIFAIR. Can you explain that? I would recommend including this information in the main paper.

Author response:
The Maximum Output Score, follows the same concept as the Maximum Logit Score. A different name was used as DIFAIR does not output logits but rather an image representation. The maximum value of the representation is selected and used as the OSR score. A high score would indicated the detection of a known class, with a highly activated attribute, while a low score signifies that no known class was detected.

Reviewer comment (Novelty):
The paper is missing large bodies of related works on metric learning, which learn multiple linear sub-spaces for each class. For example: "SoftTriple Loss: Deep Metric Learning Without Triplet Sampling," Qian et al. 2017, ICCV, and many follow-ups.

Author response:
Thank you for providing those references. We will examine those fields and papers more closely.

Reviewer comment (Experiments):
Section-3.3 does not offer much (if any) value. It would be better to provide more experimental results, given that the current ones in Table-1 are already discouraging. An analysis on interpretability of representations could be an option.

Author response:
As you and other reviewers have pointed out, further investigation into the interpretability of representations is necessary. Mainly, we will provide results on datasets other than CIFAR10, with larger representations and more classes.

Reviewer comment (Novelty):
It seems that the proposed method is only enforcing data representation to be within the spherical area around anchors of each class. Similar ideas have been explored in previous works such as [1] on other topics. Therefore, the author have to justify the extra insights provided by their work.

Author response:
Regarding point 1, as far as I understand it at the moment, the approach in reference [1] uniformly distribute the representation of classes on a hypersphere centered at the origin of the representation space. Additionally, instances also appear to be represented on this hypersphere and must maintain proximity to the defined centers. 

In DIFAIR, the defined anchors are located on a hypersphere centered at the origin of the representation space, but they are not uniformly distributed on the hypersphere. These anchors are targets for the representations, where each dimension represents extracted features. The activation of the values on dimensions depend on the presence of a features in the input image. In the DIFAIR approach, the allocation of a hypersphere around the anchor allows attributes from other classes to be activated to maintain a semantic meaning in the representation. Furthermore, our approach provides more interpretable representations by associating each dimension of the representation with a class.

Reviewer comment (Theory):
As far as I am concerned, DIFAIR only introduces an optimization objective to enforce representations being compact around anchors of each class. How can the method enforce the representations of unseen classes to be out of the spherical areas of the known classes?

Author response:
The proposed method does not directly enforce the representation of unseen classes to be outside the hyperspheres of known classes. The approach is based on the hypothesis that the values expressed on each dimension of the representation will represent the presence or absence of a feature in the image. This is due to the fact that when features of other classes are activated, the instance will be represented too far from its anchor. Therefore, if an image contains a known class, a maximum of features of that class should be detected and no features of other classes. This will ensure that the representation of the image will be placed near the corresponding class anchor (in the hypersphere). However, in the situation where an image contains an unknown class, no known features (learned on known data) should be detected, therefore placing the image out of allocated areas.

Reviewer comment (Experiments):
DIFAIR's fundamental premise is that each representation dimension corresponds to a specific class. To accomplish this, DIFAIR trains an anchor for each class, ensuring that only certain dimensions are activated for that class. While this might enhance Open-Set Recognition (OSR) and model interpretability, it raises the question of whether such a model could maintain the strong closed-set classification performance of conventional deep visual models, particularly when trained on large-scale datasets like ImageNet. The authors suggest that the hypersphere around each anchor provides some tolerance for activating other class features, but there seems to be a trade-off between interpretability (requiring low tolerance) and closed-set classification performance (requiring high tolerance). It's unclear if DIFAIR can easily find a balance point that provides both satisfactory closed-set classification performance and good interpretability.

Author response:
Thank you for your time and feedback.  

1.  To be precise, we do not train an anchor (as other approaches like DCHS [1] do), but we fix it in advance so as to bias the training process toward building representation within a certain space around the pre-defined anchors. In Table 1, we refer to Appendix D, where we show that the closed-set performance of DIFAIR is similar to the closed-set performance of a model trained with cross-entropy. But you are right in saying that it should be tested on larger scale datasets to see if the trend holds. \
Regarding the tolerance provided by the hypersphere, I would rather say that there is a trade-off between semantic in the representations and open-set classification. To have more semantic meaning, some attributes must be shared between classes. The radius of the hypersphere defines how tolerant the loss is to the activation of other classes features. However, if the tolerance is high (larger radius), class instances can be represented far from anchors, which is the behavior excepted for unknown instances. This can result in a lower open-set performance.                   

[1] From anomaly detection to open set recognition: Bridging the gap, Cevikalp et al., 2023

Reviewer comment (Theory):
The paper lacks essential theoretical support and analysis. Many ideas seem to spring from intuition. While hypotheses and assumptions are discussed in Section 3.3, and some are empirically verified, none of them is well theoretically supported.

Author response:
2.  It seemed hard for us to theoretically support our hypothesis, we developed intuitions on widely accepted hypothesis on neural networks like the extraction of features by CNNs. We will try to find more references for the basis of our reflections. Do you have any ideas or references in mind that provides theoretical support on subjects such as interpretability or semantic in representations?

Reviewer comment (Novelty):
The technical contribution is limited. The main contribution of this paper is a loss term, which however has not been extensively proved effective for a broad range of applications and for various methods.

Author response:
Thank you for your time and feedback. 

You are correct in stating that the loss term should be extensively tested on other applications and datasets. However, our goal was to present our first attempt to accomplish the objectives we set. Specifically, to train the network to obtain a more interpretable representation by associating dimensions with classes, while also attempting to retain some semantic meaning.

We believed that the proposed loss term (Euclidean distance with a threshold), which seemed to be the simplest approach to distribute instances within hyperspheres around anchors in the representation space, could achieve these goals. The tests carried out on the OSR task and the representation visualizations allowed us to identify drawbacks of our approach. Therefore supporting the importance of being able to extract information from the representation.

Reviewer comment (Experiments):
The evaluation is not comprehensive. Experiments only on small small-scale datasets are conducted, making the evaluation less convincing. Results on large-scale datasets should be conducted and more extensive analysis should be performed to have a more comprehensive evaluation.

Author response:
In this situation, we were not yet interested in testing on a larger scale, but we preferred to clearly state our hypotheses and present what needs to be addressed in future work to achieve our goals.

In your comment : 

 > Results on large-scale datasets should be conducted and more extensive analysis should be performed to have a more comprehensive evaluation.

I did not quite understand what you meant by ``more extensive analysis’’. Did you mean an analysis of the representations? Or were you referring to testing on other applications and datasets?