Paper: Characterizing Exceptional Distributions with Neural Rule Extraction (ICLR-2024)

Reviewer comment (Theory):
Many of the quantities used in the derivation are not defined. The quantities \(p_Y\), \(p_{Y|S=1}\), \(p_{Y|S=1, \mathbf{x}}\), etc. are presumably densities and conditional densities of these random variables with respect to the Lebesgue measure, but these are never defined explicitly. They are sometimes used interchangeably with the probability laws themselves, e.g., the authors also refer to \(D_{\mathrm{KL}}(p_{Y|S=1}\|p_Y)\).

Author response:
In our work we do introduce the used quantities as densities, but leave implicit the measure-theoretic formalisation as it is not necessary for the needs of this work. As for the divergences, indeed, previously we slightly abused notation to avoid the introduction of the laws, again, as these are not used elsewhere in the manuscript. To avoid confusion, however, we now introduce this notation in the updated manuscript at the end of the preliminary section (Sec 2), so that we can now adopt the usual notation.

Reviewer comment (Experiments):
It is suspicious that the number of features which actually need to be thresholded \(f_p\) is so small compared to the ambient dimension in the experimental results. What happens when a more significant fraction of the features need to be thresholded? Also, I assume the information about which features must be thresholded is not being supplied to the algorithm, but this isn't stated explicitly.

Author response:
Number of features in synthetic data  
There are two main reasons behind the maximum of 10 features which define the subgroup. Firstly, the premise of our work (and subgroup discovery in general) is that the results have to be announced to a human, and therefore fewer features lead to more succinct descriptions and are therefore considered superior, from the interpretability perspective.  
The number of 10 was chosen as, intuitively, we considered it as a round number that was already past the limit of what could reasonably be interpreted by a human expert (e.g., doctor).  
Secondly, increasing the amount of features for thresholding also comes with a data generation problem. Imposing a rule with 100 conditions on independent features would lead to an interval that covers the entire feature domain ($0.2^(1/100)≈1$), or if the features are correlated, extreme redundancy in which feature to actually threshold. This would allow a different process to start becoming apparent, in which arbitrary features would simply become relevant simply by random chance, and necessitate additional mechanisms for testing (e.g., statistical corrections for random correlation).

Reviewer comment (Evaluation):
For the real-world data experiments, reporting the size-corrected KL divergence is basically reporting the value of the objective that SYFLOW optimizes, so it is unsurprising that SYFLOW has the best performance. Including some other metrics like deviations of common statistics (mean, standard deviation, etc.) from their population levels would be a more convincing comparison.

Author response:
KL vs other metrics  
We agree with the reviewer that comparing to methods that optimise different objectives is inherently difficult. However, KL divergence is a more general measure of distribution dissimilarity than other widely used criteria; for instance, when assuming Gaussian target distributions, then there is a monotone bijection between KL and difference of means, while in other cases we could easily craft a case where the KL divergence exceeds an arbitrary bound, while the mean shift remains close to zero.  
In other words, a low mean difference does not imply that the resulting subgroup is not interesting, as in the exponential wage distribution example from Fig. 3b.

Reviewer comment (Experiments):
The authors also don't describe or give explicit reference to the baseline methods.

Author response:
Description of baselines  
We fixed this in the revised manuscript.

Reviewer comment (Novelty):
There are some missing references that should be discussed, including:  
1. Christopher Sutton, Mario Boley, Luca M Ghiringhelli, Matthias Rupp, Jilles Vreeken, and Matthias Scheffler. Identifying domains of applicability of machine learning models for materials science. Nature communications, 11 (1):1–9, 2020. (Relevant for the COVID example.)  
2. Zachary Izzo, Ruishan Liu, and James Zou. Data-driven subgroup identification for linear regression. ICML, 2023. (Another method which does subgroup discovery without pre-specified discretizations.)  
3. What is the citation for the "most related work of its kind" described in the last paragraph of Section 4.1?

Author response:
1. , 2. We have added the suggested references to the introduction.  
3. That would be the Proença et al. We rephrased the relevant sentence to make clear that we were still referring to the last mentioned method.

Reviewer comment (Writing):
What is the precise definition of the random variable \(S\)?

Author response:
Please see II. This has been incorporated in the revised manuscript in preliminaries (last sentence) and Sec. 3.1 (first sentence).

Reviewer comment (Reproducibility):
None of the functions in Algorithm 1 (Appendix C) are defined. Can the authors provide a complete, precise definition of their algorithm? It's especially unclear how normalizing flows will be used to learn \(p_{Y|S=1}\) in a way that keeps nonzero gradients with respect to the learned boundaries. Also, what is the regularization term being used in the algorithm?

Author response:
Algorithms  
Briefly, we first fit a normalising flow to the entire data, then initialise $p_{Y|S=1}$ to the former model, and perform SGD steps that jointly update both the conditional target model and the membership rule. To avoid the gradient of the rules from vanishing, we follow a simulated annealing scheme, by starting with a very moderate temperature parameter.

Reviewer comment (Experiments):
How does the method handle discrete features, as it does in the COVID-19 experiment?

Author response:
Although our model is primarily aimed at real features, ordinal and binary features can also easily be incorporated, by simply treating the values as real. In our experiments we treat the features as real numbers, resulting in logistic-like regression. For discrete data we resort to one-hot encoding, where we create a binary variable that indicates the presence/absence of each discrete value. This allows SYFLOW to easily select individual values for its rules and matches the ability of standard subgroup discovery methods for dealing with discrete data.

Reviewer comment (Theory):
The paper uses a latent variable $S$ to represent subgroups without providing a clear definition of it. An unclear aspect of this problem definition is how the latent variable $S$ is determined. While the paper treats the latent variable $S$ as a variable related to $X$ and $Y$, the assumption regarding this relationship is not explicitly stated. However, estimating the latent variable without assumptions is evidently impossible. Consequently, it is possible that there are implicit assumptions in the problem definition of this paper that have not been explicitly mentioned.

Author response:
Thank you for your detailed feedback.
1. The latent random variable S was introduced in the manuscript in the last part of Sec. 2. as a (latent) binary random variable representing the membership of a sample to the subgroup. We see that this is both terse and easy to miss. We therefore now discuss its role in the model and during inference in more detail in the beginning of Section 3.1 of the updated manuscript.

Reviewer comment (Experiments):
In the experiments, several methods such as SD-$\mu$, SD-KL, RSD, and BumpHunting are introduced as the baseline methods to be compared with. However, these baseline methods are introduced without references, making it impossible to determine what these methods do and whether they are valid as baseline methods.

Author response:
Thank you for your detailed feedback.
2. We now give an introduction to describe the baselines in the beginning of the experiment section.

Reviewer comment (Reproducibility):
Details of the experimental setup for the proposed method are not provided in the paper. For instance, how the temperature parameter $t$ was set and decayed, and how the size-corrected KL's $\gamma$ was configured, are not described. These parameters are crucial for achieving favorable results with the proposed method, both in this experiment and for practical applications.

Author response:
Thank you for your detailed feedback.
3. We now describe the hyperparameters, data generation and training procedure in the Appendix of the revised manuscript.

Reviewer comment (Reproducibility):
The paper does not describe the methods for setting various parameters, such as optimization algorithms, their parameters, and the number of training epochs, which are essential to ensure the reproducibility of the results.

Author response:
Thank you for your detailed feedback.
3. We now describe the hyperparameters, data generation and training procedure in the Appendix of the revised manuscript.

Reviewer comment (Theory):
There are two limit parameters to set the range for the subsets, what if the subsets are multi-modal? Are multiple ranges applied to each feature? I.e. to capture both a subset with young adults and another with older individuals?

Author response:
We thank the reviewer for their kind feedback. In regards to the posed questions:  
1. Currently, a subset is limited to one continuous interval per feature, so that we would need two subsets to capture two ranges right now. Our newly introduced regularizer helps find such contrasting subsets.

Reviewer comment (Writing):
Cutting the abstract into three paragraphs makes it easier to read since it is so long but I suggest to the authors that they solve it instead by making the abstract more concise, in one paragraph.

Author response:
Further writing such as making the abstract more concise and updating the captions are underway and will be incorporated as soon as possible. Thank you very much for your helpful feedback.

Reviewer comment (Presentation):
The figure captions could be more descriptive to help the reader understand the context better, also, the captions could be a *little* bit larger.

Author response:
Further writing such as making the abstract more concise and updating the captions are underway and will be incorporated as soon as possible. Thank you very much for your helpful feedback.

Reviewer comment (Experiments):
Identification of a single subgroup versus the rest of population might of low practical utility, with comparison to identify multiple subgroups that are different from each other.

Author response:
Although it may be interesting to find multiple subgroups, we note that it is not our main objective.  We here focus on developing a method that can: 1) deal with arbitrary target distributions, 2) find symbolic and highly interpretable descriptions of numeric features in a differentiable end-to-end fashion and 3) significantly reduce the runtime compared to state-of-the-art baselines.  
Nonetheless, our method can be extended in a straightforward manner to find a diverse set of subgroups. The main idea is to iteratively search for subgroups and extend the optimization objective with a regularizer of the form $KL(P_{Y|S} || P_{Y|S_{i-1}})$, described in Appendix C. This allows us to find subgroups that differ from the overall distribution and simultaneously differ from previously found subgroups.

Reviewer comment (Evaluation):
Identify distribution-wise separable groups might be meaningful from statistical perspective, but might not be meaningful from real-world view (e.g. rule of ages might be of highest KL-divergence in identification of subgroup A, but the subgroup of the 2nd highest KL based on gender might matter most in clinical perspective). Also, the quantification measure of divergence might result in potential over-fitting on existing data and cannot be generalized to unseen data. How to ensure robustness of the proposed method?

Author response:
We show in the covid experiment the ability of SyFlow to adjust for factors which may be not clinically relevant, by means of searching the joint distribution of target quantity (e.g. survivial) and factor to adjust for (e.g. age). This advantage is unique to SyFlow, as it is the only method that is designed for non-gaussian multi-variate data.

Reviewer comment (Experiments):
Not sure whether experiment results are cross-validated in Figure 3.

Author response:
They are not. Our goal is description, rather than prediction, and hence cross-validation does not make much sense. After all, the point in description (unsupervised) is to provide insight into the available data, rather than achieving the best possible generalization to unseen data, as is the point in predictive (supervised) settings. To show that our method discovers subgroups with exceptionally distributed target variables, we report the size-corrected KL-divergence of the top subgroup per method

Reviewer comment (Presentation):
It's not clear how the characteristics of subgroups identified in different models differ from each other in Figure 3. It will be helpful to review the rules similar to Figure 4, but compared across different models.

Author response:
This is a fair point. We will include the discovered rules in the Appendix. Note, however, that these results only make sense when the discovered subgroups have meaningfully high divergences; for wine and wages, for example, the competing methods all fail to discover a subgroup with an exceptional distribution of the target variable, so it makes little sense to inspect these rules.

Reviewer comment (Presentation):
The paper lacks the description of limitations in final conclusion part.

Author response:
We added a Limitations section to the Appendix, and propose to prune some of the figures in the experiments to make space for this in the main manuscript.

Reviewer comment (Evaluation):
How to validate the robustness of the proposed approach?

Author response:
We test SyFlow across a variety of data distribution types as well as on a diverse set on real world datasets, where it is consistently amongst the top methods.

Reviewer comment (Theory):
Will the proposed approach guarantee best classification if subgroup is known?

Author response:
We are not exactly sure what you mean by this, but in our experiments SyFlow recovers a planted ground truth subgroup (interpretable as class) accurately.

Reviewer comment (Theory):
In Section 3.3, the divergence (eq 2) is first approximated using eq 7, and subsequently eq 7 is approximated via sampling. Why couldn't the eq 2 be directly approximated via sampling?

Author response:
Eq (2) does not involve the subgroup rule function s, while Eq (7) does and therewith gives us gradients to optimize the rule s.

Reviewer comment (Evaluation):
The evaluations are somewhat inadequate. First, what is SD-\mu in Section 5.1? Second, what are the cutpoints there? Finally, what are the various columns in Table/Figure 3 (i.e. where are these methods defined)?

Author response:
We have added a paragraph describing the baseline methods to the updated manuscript.

Reviewer comment (Experiments):
The paper mentions on page 2 that `As we show, this quantisation can greatly reduce the quality of the results` -- where is this shown?

Author response:
The effect of quantisation is shown in the first experiment of the synthetic experiments (Feature Thresholding), where we vary the number of cut points for subgroup discovery (SD-$\mu$). The results are displayed in Figure 1.b. For a small number of cutpoints (coarse quantization), the performance is significantly worse than for a finer quantization. To match the performance of SyFlow, SD-$\mu$ requires 40 cutpoints, while the runtime increases by three orders of magnitude.