Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This work studies designing interpretable geometric deep learning via learnable randomness injection with 4 proposed datasets, ActsTrack, Tau3mu, SynMol, and PLBind.  They show that using LRI mechanisms could improve the prediction performance of the backbone models, and even could improve model generalization.
strength_and_weaknesses: Strength:
1. It is interesting to study using interpretable learning mechanisms for point cloud data, especially for proteins that often consist of hundreds of amino acids.
2. It is also good to establish 4 benchmark datasets to facilitate interpretable GDL research.
3. Their experimental results show appealing properties.

Weaknesses:
1. The paper is not well-organized, and also is not easy to read.  I would suggest moving the dataset introduction part to a later section, and could better prepare a full system overview to better illustrate how the full system work. 
2. According to Table 1, all 4 datasets seem to be not large-scale, and each dataset only has 2 classes.  I am not sure if that is sufficient for a classification benchmark.  The number of points in each sample seems to be not large either (i.e., the largest is 339.8), which should be much less than a normal visual point cloud scan (e.g., at least 1024 in ModelNet40).  Maybe the authors could explain more in the response.
clarity,_quality,_novelty_and_reproducibility: Frankly speaking, I am not an expert in protein analysis, and I feel this manuscript is not easy to follow.  It is welcome to conduct interdisciplinary research, but I strongly suggest the authors could highly improve its readability.
summary_of_the_review: At the current stage, I think it seems to be interesting research.  I would be willing to reconsider when the other review and the author's responses are available.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a framework for Learnable Randomness Injection (LRI) to train inherently interpretable geometric deep-learning models for scientific data. It perturbed data by learnable Bernoulli and Gaussian randomness masks in terms of existence and location importance. Furthermore, four scientific datasets with ground truth interpretation are proposed. 
strength_and_weaknesses: Strengths:
+ The paper is well-organized, written, and easy to follow.

+ The proposed research is well-motivated and appealing.

+ The proposed methods are simple and technically sound.

+ Four scientific datasets with interpretation ground truth will be released, which will benefit a large research community.

Weaknesses: 
- Insufficient justification of technical contributions. Although the proposed methods, namely LRI-Bernoulli and LRI-Gaussian, are technically sound, it is hard to tell the technical contributions compared with existing works. For instance, PointMark, BernMask (GNNExplainer), GradGeo (DeepLIFT) are intensionally selected for comparison. How are they related to the proposed methods? What are the key differences/improvements from the proposed methods and previous mask-based methods (e.g., PointMark)? 

- Computation overhead. Constructing KNN graphs is very time-consuming. The proposed LRI-Gaussian has to perform graph reconstruction after each round of perturbations. However, there is no discussion about computational cost either analytically or empirically.

- Missing ablation study. Without ablation studies, it is hard to tell whether the prediction and interpretation gains result from more parameters and additional computations.
The authors claim that existing interpretation methods cannot handle irregular geometric data. However, there is limited evidence to show that the proposed methods can handle it. 
The authors claim that most existing methods can only provide either the existence or location importance. However, is this really a significant problem? 
Can the proposed LRI-Bernoulli and LRI-Gaussian work together? 

Minor Issues:
- The structures of g and f is unclear, how the f take points and 3D spatial coordinates to make predictions? 
- How is the $| \Sigma_v|$ is used to rank the location importance?
- How the eigenvectors of $| \Sigma_v|$ is calculated?

clarity,_quality,_novelty_and_reproducibility: The paper is well-written, and it is a joy to read, especially the introduction. The proposed methods are clearly presented with sufficient details. Overall, it is a well-prepared submission with sound quality in clarity, quality, and reproducibility. 
summary_of_the_review: My major concern is the insufficient justification of technical contributions, computational cost analysis, and missing ablation study.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The submission develops two setups for building interpretable classifiers for geometric data and carefully applies them on four different test scenarios (from physics and medicine). The proposed models performs sensitivity analysis wrt either point inclusion (Bernoulli model) or point perturbation (Gaussian model). The proposed models are simple and can easily bee integrated with existing classifier models. The proposed models and classifier backbones are jointly trained. The paper also contributes benchmark datasets by collecting data from pretty diverse application settings and demonstrates the utility of the proposed framework. 

I liked this submission as it demonstrates applications on non-standard dataset with potential implication beyond our immediate field. 
strength_and_weaknesses: + Important topic and a simple clean framework
+ End-to-end trainable wrt interpretability and classification
+ Benchmark datasets and good evaluation

The following are some of the weaknesses that can be explored in future works. 
- Correlation across points is not considered. In many situation, this can be quite an important factor. 
- Performance in adversarial setting is not discussed/explored. 
- GradGAM, one of the gradient-based baselines, turns out to be a pretty good post-optimization approach. 
clarity,_quality,_novelty_and_reproducibility: The paper is well written and illustrated. The evaluation protocol is succinctly explained and the dataset gathering effort is appreciated by the reviewer. The work should be reproducible and easy to integrate. 

Equation 2 can be simplified by penalizing deviation of the covariance from identity as the distributions are Gaussian? 
Is the effect of \beta parameter discussed in the paper? 
summary_of_the_review: Overall a good paper and I would be supportive of acceptance.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper proposes two training schemas for fostering interpretability in Geometric Deep Learning by considering two properties: the relevance of a point presence and the relevance of a point location. A backbone classifier network is trained in an adversarial way where an Interpreter modifies the input data with a given amount of point-wise noise (either by removing points or perturbing their position). Then, the classifier module tries to improve the score by removing the noise while the Interpreter attempts to redistribute the amount of noise. In the end, the noise should be located at non-relevant points, while important points are kept unchanged. The method is tested across four different datasets from physic and chemistry, outperforming baselines in most cases.
strength_and_weaknesses: STRENGHT
=========
S1) NOVELTY AND RELEVANCE: While similar principles have been proposed in the past to study neural network interpretability (on graphs: Miao et al. 2022; on images: "U-noise: Learnable Noise Masks For Interpretable Image Segmentation", Koker et al., 2021), I am not aware of similar approaches in the field of 3D point clouds. The formulation looks straightforward and applicable also in contexts outside the datasets proposed in the paper. Improving interpretability in Geometric Deep learning is an active research field, and I believe the proposed methods might have a relevant impact.

S2) EXPERIMENTS: The two approaches are carefully tested with different backbones networks on several datasets, and also provide ablation and explanations of hyperparameter choices. 

WEAKNESSES
========
W1) LIMITATIONS AND DISCUSSION: The paper does not offer an overview of methods limitations, nor discussion or visualization of the achieved interoperability for the different tasks. Some sentences are spread across the text, pointing to how to exploit the covariance matrix to obtain fine-grained geometric patterns, but without showing examples. Clarify the interpretability analysis and clearly stating the outcomes could strengthen the paper message.

W2) GEOMETRICAL DIVERSITY OF THE DATASETS: The proposed datasets present similar characteristics. The number of points for each sample is similar (between 15 and 340), with a similar proportion of importance points (between 20% and 30% of the entire point cloud) and the same number of classes (2 for all the datasets). It is also not clear if the different datasets face different points distribution (i.e., the density of the point clouds). I would suggest testing on data from different domains, coming from Computer Vision and Graphics (e.g., LiDAR, body scans, ShapeNet rigid objects), and also providing a measure about the density of the points w.r.t. the occupied space. 


MINOR
=======
M1) Can LBI-Bernoulli and LBI-Gaussian be trained together? Does it introduce instability in training? I think that if no technical limitations exist, it would be an interesting validation.
M2) I think the methods can potentially work also on other domains (e.g., images, graphs), up to a minor reformulation of the two strategies. Is it correct? In that case, it might be mentioned as possible future work.
clarity,_quality,_novelty_and_reproducibility: I find the method clear, while some part of the specific tasks and the reported outcome seems not entirely accessible for non-expert readers. I consider both Novelty and Reproducibility positively. The authors attached the code, but I do not see statements about code and data release.
summary_of_the_review: While some aspects in the discussion and experiments of the results could be improved, I have a positive opinion of the work: the contribution is interesting, well-validated, and based on straightforward principles.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper studies interpretable geometric deep learning by designing the learnable randomness injection (LRI) mechanism.  It also proposes four datasets from real scientific applications to evaluate the proposed LRI mechanism. Experiments show that the LRI mechanisms can improve the prediction performance of the backbone models, and even improve the model generalization. All reviewers agree the research problem in the paper is important and interesting, find the proposed methods to be technically sound, and think the four collected benchmark datasets are valuable.  After the rebuttal, all 4 reviewers lean to accept the paper (particularly, 2 of them champion the paper, with each rating a score 8: accept). The rebuttal has addressed all most of the concerns raised by reviewers, including the readability of the dataset description, missed justification of technical contributions, and ablation study. The AC agrees with the judgements of the reviewers and recommends accepting the paper.   AC urges the authors to improve their paper by taking into account all the suggestions provided by the reviewers and including all additional experiment results into the revision.

justification_for_why_not_higher_score: Not all of 4 reviewers champion the paper. Two of them just vote for marginal accept. Considering the quality of the current version, the AC recommend it to be accepted as a poster.  
justification_for_why_not_lower_score: The paper makes contribution on new method and new datasets. It is a valuable paper worthy to publish.
Decision: Accept: poster