Venue: CLeaR 2022 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: Summary: This paper is concerned with learning interpretable deep representations by combining DNNs with decision trees. 
Main review: The paper proposes a clever and simple-to-implement way of learning interpretable representations for deep NNs, by jointly training them with decision trees. This question is relevant for the CleAR community since learning these types of representations is central to integrating causality with deep learning. 

The work is technically sound, and clearly written. Related work is discussed in detail and the problem is well-motivated. The empirical experiments are convincing, and they are a nice demonstration that superior performance and interpretability do not necessarily go hand in hand. The ablation experiments also help strengthen the paper. 

My only criticism is that when the implementation of the algorithm is presented, it is not clearly stated exactly which part of their pseudo-code and objective function corresponds to intersection regularization. Furthermore, it would be great if the authors can comment on whether the intersection regularization is actually necessary, if they were to use exactly the same architecture they proposed, and whether they have performed any experiments with that?
review_title: null

final_score: rating: 6: Marginally above acceptance threshold
review_content: Summary: This paper proposes a novel problem named weakly supervised discovery of semantic attributes, which aims at extracting the semantic attributes based on simply image-level labels. To address this issue, this paper proposes to take advantage of MLP and decision tree to optimize the main classification network. Besides, Intersection Regularization is also proposed for training the MLP the decision tree together, and some relevant empirical results and theoretical analysis are also provided.
Main review: This paper proposes a novel problem named weakly supervised discovery of semantic attributes, which aims at extracting the semantic attributes based on simply image-level labels. To address this issue, this paper proposes to take advantage of MLP and decision tree to optimize the main classification network. Besides, Intersection Regularization is also proposed for training the MLP the decision tree together, and some relevant empirical results and theoretical analysis are also provided.

Pros:
1.	The paper makes a concrete and comprehensive overview of relevant work, especially in the field of Interpretability.
2.	The paper proposes an interesting method to solve the proposed issue. The MLP behind the feature map is proposed to represent the quantized function for semantic attributes. The decision tree is proposed to auxiliary the main classification network to learn the relationship from attributes to image-level labels. The idea is easily implemented, and such modeling for these two different signals (semantic attributes and classification labels) is reasonable. 
3.	This paper also proposes a method named Intersection Regularization to optimize the main classification network by interactively updating the MLP and decision tree, which is also proved to be effective by some experiment results.

Cons:
1.	There are some detailed errors in writing. For instance, page 5 mentioned “i’th elementary vector” and “ith attributes” at the same time. Besides, some nouns and sentences need to be clarified and defined so as to eliminate the ambiguity in this paper. For example, “feature fidelity” in Section 3.1, ”GD” in Section 5, and why “U = R, [-1,1] or {±1}”. 
2.	There is a confusing definition to the newly proposed problem. The title is “Weakly supervised Discovery of Semantic Attributes”, but page 2 also defines it as “Weakly Unsupervised Discovery of Semantic Attributes”. Therefore, I suggest the author should make an accurate name or definition to the proposed problem.
3.	The motivation of the proposed problem seems not understandable to me. The essence of weakly supervised/unsupervised learning is to liberate humans from exhaustive annotation work, which means that the network could work with fewer or even no labels. Although this paper aims to find the attributes without annotating them, the image-level labels are utilized during the training process. It may sound more reasonable to utilize noisy or fewer labels for semantic attributes during the training phase so that the problem could be defined as “weakly supervised discovery of attributes”. 
4.	The meaning of exploring the potential semantic attributes should serve to better classification. However, the experiment results show that the classification accuracy of proposed method is way lower than some baseline methods, it may not convincing that only the evaluation to the recovered attributes achieves higher performance. Besides, the sole evaluation metric is feature fidelity score, which is newly proposed in this paper. Therefore, the author should show more solid and fair experiment results.
5.	Lack of detailed implementation about the experiments including but not limited to the relevant data augmentation and training hyper-parameters such as batch size.
review_title: null

final_score: rating: 6: Marginally above acceptance threshold
review_content: Summary: The paper proposes to learn an interpretable discrete layer with both quantization and regularization with decision trees.
Main review: Strengths:
The results appear promising: according to the newly defined measure of feature fidelity, the proposed model learns more interpretable discrete representations without much loss on the performance.

The idea of hypothesis-space regularization is interesting (but it could be better motivated, see below).

The set of baselines and datasets makes sense.

Weaknesses:
It is not clear that the paper fits well with the causal learning and reasoning theme. While learning discrete representations is likely to be useful in the field of causal ML, the paper does not discuss these connections.

The paper does not motivate the intuition behind the hypothesis-regularization idea. Why should we expect it to work? How the proposed training dynamic is expected to enforce feature fidelity in the discrete representation.

An important aspect of the approach is the training dynamic arising from alternating between updates on the two losses and the tree retraining. But the experiments do not show this dynamic, they only focus on the final outcome.

The number of dimensions in the discrete layer seems to be chosen much larger than the annotations used for evaluation (256, 512 in networks versus 64, 50 in datasets). How does the proposed model behave when the number of dimensions is the same or smaller than in the annotations? In Figure 3. some ablation is performed but the number of dimensions still remains >= 128.



Minor comments:
In the computation of feature fidelity: what strategy is used to compare 4-bits attributes with binary attributes, or to compare with numeric attributes?

In section 3.1
max{r(q1, q2;D), r(q1, 1 − q2;D)} shouldn't it be 1 - q1 as well in the second term to have invariance w.r.t. whether the positive are marked as 1 or 0.
review_title: null

Meta-review: This paper proposes a simple approach to learn interpretable representations. The approach is well positioned with respect to prior work and all reviewers found the paper interesting, well written, and technically sound.

It should be presented at the conference and will be of interest to the causality community.
Decision: Accept (Poster)