Venue: NeurIPS 2022 Workshop TEA

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: Summary:
In this paper, the authors proposed a Sparse Low-Dimensional Decision "SLDD-Model" for image classification. In particular, the "SLDD-Model" ensures a low-dimensional feature space, by performing feature selection prior to the last (linear) layer of a neural network, and the model ensures sparsity, by using glm-saga to solve for the last (linear) layer of the network. After feature selection and last layer sparsification, the feature extraction backbone of the network is fine-tuned to adapt to the low-dimensional feature space and the fixed sparse last layer. To understand the meaning of the selected features, the authors used posthoc feature alignment with human interpretable concepts.

Strengths:
- The paper provided a systematic way to obtain a low-dimensional feature space and a sparse linear last layer, which ensures that the network is only using a handful of features in the final classification layer.
- The experimental evaluations are comprehensive.

Weaknesses:
- There is not much technical innovation in this work. The authors are just using existing techniques for feature selection and sparsification, and applying them to the last layer of a neural network.
- There is a need for posthoc feature alignment with human interpretable concepts, in order to understand the features learned/selected. However, as the authors acknowledge, not all features can be aligned with human interpretable concepts. This makes the authors' claim that the network is "interpretable" rather weak (because a low-dimensional feature space and a sparse last layer are not enough for interpretability, unless the features themselves are interpretable).
- The literature review is lacking. Since the authors cited the ProtoTree (Nauta et al., 2021), I am curious as to why the authors ignored a large body of prototype-based interpretable classifiers (e.g., ProtoPNet (Chen et al., 2019), ProtoPShare (Rymarczyk et al., 2021), ProtoPool (Rymarczyk et al., 2022)). These classifiers provide interpretable features through visualizable prototypes, and also make attempts to sparsify the last layer for interpretability.
- Finally, a 2048-dimensional feature space used in some experiments is still a high-dimensional feature space.
review_title: A Paper on a Sparse Low-Dimensional Neural Network for Image Classification

final_score: rating: 6: Marginally above acceptance threshold
review_content: This work proposes the SLDD-Model, which reduces large, unstructured inputs (e.g., images) down to a smaller subset of semantically meaningful features. A smaller classifier then operates over the semantic features, producing classifications that are attributable to a smaller set of human-readable features.

The method is interesting, the results are strong, and I am interested to see the method applied more broadly. My primary concern lies in the clarity of presentation for the method, and the lack of open-sourced code. I do not believe it would be simple to reproduce the results of this work using only the equations and descriptions given.

Strengths:
* The results of the SLDD-Model are strong, the discovered features are interesting, and the qualitative examples are sensible.
* The relative strengths and weaknesses of saliency-based approaches to interpretability are discussed well.
*  The experiments are thorough in comparisons to prior work for fine-grained classification.

Weaknesses:
* The method's clarity is lacking. Glm-saga seems like it should be a preliminary or mentioned earlier, as it is referenced in the main body of the method but then introduced rather late.
* The contribution of this paper relative to baselines is not clear. It appears to be a combination of the feature diversity loss and a feature-selection method, but the feature-selection method is not ablated anywhere (the diversity loss is ablated in Table 3 I believe)
* Ablations are present for performance on benchmarks, but not for interpretability. It would be nice to see how the diversity loss affects the learned features, for example.

I would like to see a clearer statement of contributions for this work relative to prior work (much of the paper is closely tied to prior work, such as glm-saga, but the distinctions are not made clear in many places outside of Equation 3) and a clearer ablation of which components of the method contribute to interpretability. It would also be nice if the work were more easily reproducible. Regardless, the results are strong and the method is interesting.
review_title: Interesting Method with Strong Results

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: This paper argues that because humans can only consider around 5-10 aspects at a time when making a decision, models used for interpretability should similarly be sparse. They propose a model that, for increased interpretability, has a very sparse last layer (5 neurons) which should make it feasible for a person to visually analyze each feature and understand its meaning. I really liked this paper and this it's a great fit for the workshop, well done! For future work, I'd love to see a user study that investigates how interpretable these sparse features are to real people. I'd also like to see a discussion on how people being able to interpret these features can have a positive impact on embodied robot failure and behavior understanding in real users' homes.
review_title: Really nice paper on building interpretable features

Meta-review: This paper proposed an interesting method to induce a low-dimensional feature space for interpretable image classification. The proposed method is supported by comprehensive and strong experimental evaluations. Please consider the reviewers' comments in the final version. One common concern is it is unclear if the paper has novel technical innovation compared to the prior works.
Decision: Accept