Paper: Exploiting Negative Samples: A Catalyst for Cohort Discovery in Healthcare Analytics (ICLR-2024)

Reviewer comment (Experiments):
Although the authors provided an extensive qualitative analysis of the clusters discovered by the proposed method, it is limited to a single private dataset, which hinders its broader impact to other medical datasets.

Author response:
**Weakness 1**

We have extended the evaluation of our proposed cohort discovery approach to the widely recognized MIMIC-III dataset [c] in Appendix G.8. The MIMIC-III dataset serves as a prominent benchmark in healthcare analytics, collecting EMR data from dozens of thousands of patients admitted to intensive care units (ICU) between 2001 and 2012. In this particular evaluation, our focus centers on predicting in-hospital mortality, leveraging laboratory test data as input. Specifically, we consider each patient admission, representing an individual visit to the hospital, as a sample if its duration exceeds 48 hours. The mortality label for each admission is determined by whether the patient passes away during the specific hospital admission or not. Utilizing a set of 428 laboratory tests, we predict the likelihood of mortality for each admission.  A summary of the key statistics pertaining to the MIMIC-III dataset for in-hospital mortality prediction is provided in Table 3.

To validate the broad applicability of our proposed approach, we have conducted the same cohort discovery analysis on the MIMIC-III dataset as described, with the results shown in Figure 18. In particular, through the application of a Gaussian mixture model, we identify three distinct components within the data Shapley value histogram, as depicted in Figure 18(a). To focus our analysis on the third component, which holds significant relevance to the prediction task, we adopt a 50\% threshold to exclude the lower 50\% of negative samples based on their data Shapley values, while retaining the remaining 50\%. The data Shapley value distribution of all negative samples in the manifold space is visualized in Figure 18(b). Subsequently, by performing DBSCAN on the extracted 50\% of negative samples exhibiting high data Shapley values, our approach successfully discerns fifteen distinct cohorts, as demonstrated in the t-SNE plots in Figure 18(c). These fifteen cohorts effectively decompose the third component of Figure 18(a) into respective Gaussian distributions, as confirmed in Figure 19, validating consistent data Shapley values within each identified cohort. Compared to the results found in the AKI prediction, we can see that in terms of ICU mortality prediction, the negative sample cohorts are more distinct and prominent in the constructed negative sample Shapley field. This aligns with expectations, as the negative samples in mortality (i.e., cases where patients are successfully resuscitated and discharged alive) among patients on the brink of death admitted to ICU often stem from causes that are more multi-sourced and more explicit, compared to those in a single-specialty nephrology department. This substantiates the capability of our cohort discovery approach in partitioning samples with high data Shapley values into medically relevant and distinct cohorts.

The outcomes of this cohort discovery evaluation hold substantial promise in advancing our comprehension of ICU patients, and they are poised to contribute significantly to patient care and survival, particularly when combined with systematic and cross-sectoral analytical research. Our experimental results underscore the effectiveness of our proposed approach on this well-established public benchmark dataset, demonstrating its potential for broader application in the analysis of other medical datasets, thus highlighting its versatility and robustness. Further details regarding the evaluation of our approach on the MIMIC-III dataset can be found in Appendix G.8.

[c] Johnson, Alistair EW, et al. "MIMIC-III, a freely accessible critical care database." Scientific data 3.1 (2016): 1-9.

Reviewer comment (Experiments):
The key idea of the proposed work is conducting clustering on a subset of negative samples in the manifold that preserves meaningful structure about the data Shapley values. Considering such contribution, comparing it to other conventional and cutting-edge clustering methods (e.g., DCN [A], MOE-VAE [B]) will improve the clinical utility of the proposed work.

Author response:
**Weakness 2**

We have investigated the two recommended deep clustering methods: the deep clustering network (DCN) [A] and the mixture-of-experts similarity variational autoencoder (MoE-Sim-VAE) [B]. DCN achieves joint dimensionality reduction through a deep neural network and K-means clustering, and we have integrated it as a clustering baseline for comparison. MoE-Sim-VAE employs a variational autoencoder (VAE) with a decoder consisting of a mixture-of-experts (MoE) architecture. However, due to its high application specificity and numerous parameters for clustering organ-specific single-cell RNA-seq data as in its officially released implementation, we have not included it in our comparison but will discuss it in the related work section.

For a comprehensive evaluation, we have also adopted another cutting-edge deep clustering method recommended by [d], the deep embedded K-means clustering (DEKM), which transforms the embedding space further to a new space to reveal cluster-structure information.

The comparison between our proposed cohort discovery approach, DCN, and DEKM is presented in Appendix G.6, with cohort discovery results shown in Figure 16. In terms of results, neither of the newly introduced baseline methods can identify significant cohorts as effectively as our proposed approach does. Instead, they classify the majority of points into one large cluster. These results align with those obtained using cPCA in Figure 10 and our proposal without the isotropy constraint in Figure 9(b) in the ablation study. A common characteristic of these methods lies in first a dimensionality reduction of raw data using methods, e.g., AE, SAE, SDAE, or cPCA, and a subsequent clustering on the embedded representations using either K-means or DBSCAN. This indicates that the subtle differences resulting from specific implementations of dimensionality reduction and clustering methods do not lead to significant differences in cohort discovery in the application scenario.

The key difference between them and our proposed approach lies in the absence of our proposed isotropy constraint, which ensures uniform changes in data Shapley values across orientations, facilitating the identification of high-temperature areas in the field. These results substantiate the innovative contribution we have made in this paper by incorporating data valuation into medical cohort discovery. This highlights the importance of our approach, as traditional methods solely based on feature similarity struggle to effectively differentiate between different cohorts in complex and high-dimensional medical applications. This Shapley-based exploration of interrelationships between samples is our primary innovation for real-world clinical practice. 

In summary, the newly supplemented comparison results with DCN and DEKM, along with the previous results of cPCA and our proposal without the isotropy constraint, further validate the effectiveness of our proposed cohort discovery approach, underscoring its medical value.

Further, we would like to emphasize that for spatial clustering, we employ DBSCAN on account of its well-acknowledged superior performance and its capability to discover arbitrarily shaped clusters (Ester et al., 1996; Gan \& Tao, 2015; Schubert et al., 2017). It is essential to clarify that the primary contribution of our proposal lies in exploring negative samples for cohort discovery through a Shapley-based exploration of interrelationships between these negative samples. Consequently, after constructing the negative sample Shapley field, the choice of the subsequent spatial clustering method is somewhat orthogonal to our primary contribution. Therefore, we remain open to the possibility of replacing DBSCAN with more innovative and better-performing spatial clustering alternatives, should they become available, to further enhance the cohort discovery results for the benefit of clinicians.

[d] https://paperswithcode.com/task/deep-clustering

[e] Guo, Wengang, Kaiyan Lin, and Wei Ye. "Deep embedded K-means clustering." 2021 International Conference on Data Mining Workshops (ICDMW). IEEE, 2021.

Reviewer comment (Experiments):
The authors should include clustering results when clustering is performed on i) including all negative samples and ii) including positive samples to substantiate the claim of the importance of focusing on the subset of negative samples with high Shapley values.

Author response:
**Weakness 3**

We wish to reiterate that our primary objective in this work is to discover cohorts within the negative samples that contribute significantly to the prediction task. This focus allows us to facilitate clinicians' understanding of the disease from a population perspective through the insights derived from these valuable negative samples.

In response to the first suggested experiment (i), we have included a comparison between our approach's cohort discovery results among high data Shapley value negative samples and results among all negative samples in Appendix G.7. The experimental results are presented in Figure 17. Our approach, even after incorporating all negative samples for clustering, continues to identify representative cohorts composed of high data Shapley value negative samples, with only a few merging with the large cluster of low data Shapley value negative samples. It is important to emphasize that cohorts with high positive data Shapley values demonstrate a clear ability to facilitate the model to identify positive samples, indicating their fundamental value in medical research, and hence, we only focus on them in this paper. In contrast, the medical significance of cohorts without high positive data Shapley values (negative or near zero) becomes very unclear. They can either hinder the prediction task by potentially being erroneous or noisy data points, as indicated by negative data Shapley values, or they may represent complex and healthy samples of limited relevance, with data Shapley values close to zero. Consequently, these negative samples are not the primary focus of our cohort discovery efforts.

Regarding the second suggested experiment (ii), where both positive and negative samples participate in the clustering process, it is crucial to understand that our proposal, based on the Shapley-based exploration of interrelationships between negative samples, relies on the premise that valuable cohorts among negative samples should exhibit similar distributions with high data Shapley values. This premise assumes a consistent label, i.e., negative samples. When labels are mixed, including both negative and positive samples, this assertion no longer holds. Specifically, the interpretation of positive samples within our constructed negative sample Shapley field becomes unclear. Therefore, such positive samples cannot be incorporated into our approach.

Reviewer comment (Experiments):
As shown in Figure 9 (in Appendix G), the discovered clusters can be different depending on $P_{min}$, leading to different scientific discovery. How can we choose $P_{min}$ for a given dataset?

Author response:
**Question 1**

In our assessment of clustering quality for a given dataset, we rely on the Silhouette score as a metric. The Silhouette score measures both the cohesion and separation of clusters, providing a robust evaluation of clustering quality for each parameter setting. In the case of our AKI dataset, we conduct a comprehensive evaluation and determine that setting $P_{min}$ to 100 yields the most suitable parameter choice for our DBSCAN clustering, based on the Silhouette score assessment. Further details regarding the selection of parameters for the DBSCAN algorithm are elaborated in Appendix E.3.

In addition, Figure 9 in Appendix G.1 showcases how we vary the settings of $P_{min}$ to highlight the advantages of our proposal, over the variant approach without the isotropy constraint more clearly and thoroughly. As shown in Figure 9, the results of our proposal, which includes the isotropy constraint, remain relatively stable across varying $P_{min}$ settings. In contrast, when the isotropy constraint is removed, effective cohort discovery through DBSCAN is hindered, regardless of the chosen $P_{min}$ values. These results underscore the pivotal role of the introduced isotropy constraint in ensuring robust cohort discovery outcomes that are resilient to variations in spatial clustering algorithm parameters. Ultimately, this constraint contributes to the identification of medically meaningful cohorts.

It is important to note that the cohort discovery results should be viewed as preliminary outcomes for subsequent complex medical research and investigation. Clinicians will delve into the identified cohorts for in-depth analysis on a population level. Therefore, the selection of parameter settings should align with the requirements of the ensuing medical research, including considerations such as available resources and manpower, particularly with regard to how many patient groups (out of the patients of interest) need to be partitioned and analyzed for detailed investigation.

Reviewer comment (Experiments):
There are different types of clustering methods. Why did the author choose DBSCAN over other clustering methods, such as k-means or its variants? What would be the impact of using other clustering methods?

Author response:
**Question 2**

In our previous response to Weakness 2, we have already addressed this issue to some extent. Here, we briefly reiterate the key points for clarity.

Firstly, we regard DBSCAN as a mature and proven spatial clustering tool with broad applicability across various scenarios. Unlike K-means, DBSCAN does not require a pre-assumed number of clusters (K) and is capable of identifying high-density connected subspaces of arbitrary shape.

In contrast, as also noted in the reference paper [A], K-means is generally suited for clustering data samples that are evenly distributed around their centroids in the feature space. This assumption may not hold in our context of high-dimensional medical data. Furthermore, our approach seeks to identify high-temperature connected subspace in the constructed negative sample Shapley field, aligning well with DBSCAN's mechanism.

Secondly, we emphasize that our approach is not confined to a specific spatial clustering method. Indeed, the particular implementation of clustering is orthogonal to our overall cohort discovery framework. Our work is open to any other clustering methods, and the corresponding substitution is straightforward.

Finally, we stress that the ultimate aim of our work is to assist in identifying medically meaningful cohorts in clinical cases, especially in negative samples, to enhance our understanding of diseases and benefit clinical practice. In the current setup, our approach already provides insightful and valuable contributions to this end.

Reviewer comment (Evaluation):
The authors claim that negative samples with zero Shapley values typically indicate healthy individuals without any overt AKI-related symptoms. Does a high Shapley value correspond to a high likelihood of AKI? If so, why not use the predicted probability of each sample directly?

Author response:
The data Shapley value of a negative sample serves as a measure of its contribution to the prediction task. Therefore, focusing on the second component in the middle of Figure 2(a), we observe that its data Shapley values are centered around a mean value close to zero. This implies that these particular negative samples provide minimal contributions to the prediction task, suggesting that they are generally healthy without overt symptoms related to AKI. It is crucial to emphasize that the data Shapley value of a negative sample is distinct from the model's predicted probability, i.e., the likelihood of AKI. Specifically, the former quantifies the individual contribution of each negative sample, while the latter characterizes the relationship between each negative sample and the label.

Our primary objective lies in taking into account data valuation when conducting spatial clustering analysis among negative samples, and we propose to assess each negative sample based on its data Shapley value. This approach enables a Shapley-based exploration of interrelationships between the samples. To offer a comprehensive comparison, we have introduced a $p$-based data valuation strategy, where $p$ represents the model's predicted probability of a negative sample being classified into the negative class, indicating the model's confidence in the assigned label. A detailed comparison between this baseline and our proposed approach is provided in Appendix G.5.

The cohort discovery results using the $p$-based data valuation strategy are presented in Figure 14. Figure 14(a) displays the histogram of $p$ values among all negative samples, which fits well with a beta distribution. Applying a threshold of 60\%, we extract the top 40\% high-$p$ negative samples. Figure 14(b) illustrates the distribution of all negative samples in terms of their $p$ values in the manifold space. The top 40\% high-$p$ samples appear scattered throughout the space and are not clearly separated from the low-$p$ samples in dark blue. Subsequent application of DBSCAN to the 40\% high-$p$ samples results in unsatisfactory clustering outcomes, forming only two clusters, as depicted in Figure 14(c).

Additionally, we introduce a setting "$p>0.5$," which incorporates only the negative samples for which the prediction model has high confidence. We compare the performance of this newly introduced setting with our proposed approach's three different settings in terms of AUC. The results, shown in Figure 15, clearly demonstrate that "$p>0.5$" performs worse than our proposal "$d_i^-$ with $s_i>0$" and even underperforms the inclusion of all negative samples ("All $d_i^-$") for GBDT, RF, and MLP classifiers. This inferior performance is attributed to the removal of samples with low $p$ values in the "$p>0.5$" setting, resulting in the exclusion of samples that the model is not sufficiently confident about. Consequently, this exacerbates the overfitting issue, which is particularly pronounced in the MLP classifier, as evidenced by the significant degradation in AUC for the "$p>0.5$" setting shown in Figure 15.

Based on these two experiments comparing the $p$-based data valuation strategy and our data Shapley values for negative samples, it is evident that the $p$ value is not an appropriate data valuation strategy in the context of AKI prediction.

At the same time, we refer you to our response to Weakness 3(ii) of Reviewer KsRk's reviews, where we also discuss a similar concern, explaining the difference between data valuation and model confidence (predicted probability).

Reviewer comment (Evaluation):
How are negative samples defined? Do they refer to ground-truth negative samples or predicted negative samples? How are false negatives/positives addressed? Is there a chance that one of the identified clusters pertains to FN/FP cases?

Author response:
We would like to clarify that "negative samples" are cases where patients visit our nephrology department but are not diagnosed with AKI. Therefore, both the positive and negative samples in our paper are ground-truth labels, based on actual clinical diagnosis results. This work does not involve any predicted labels, and therefore, does not involve any considerations or concerns related to FN/FP.

Reviewer comment (Evaluation):
How is the clinical significance of these cohorts determined in section 4.2?

Author response:
The cohort discovery within negative samples facilitated by our proposed approach has yielded representative cohorts, the medical patterns of which are thoroughly validated by our clinicians, as demonstrated in Section 4.2. The clinical significance of these discoveries is profound. The cohorts identified among negative samples have the potential to unveil future positive cases, pathological correlations, or conditions that bear similarities. This reciprocal relationship between negative and positive samples has the potential to redefine and refine the concept of positive samples in theoretical medical research.

We would like to underscore that, as front-line clinicians and medical researchers who have focused on AKI for many years, the samples, whether positive or negative, are not merely data points within a dataset. They represent real-world patients who we have been assisting and investigating in the context of AKI in clinical practice. Consequently, the medical insights derived from our proposed cohort discovery approach deepen our clinicians' understanding of the disease, AKI, from a population perspective. This understanding has the potential to bring about substantial benefits in terms of patient management, medical resource allocation, and the advancement of medical research in the long term.

Reviewer comment (Reproducibility):
How are the predictions formulated? Are raw features, or are the learned embeddings used as inputs for LR?

Author response:
The raw features, consisting of 709 lab tests, serve as inputs for LR to calculate the data Shapley value for each negative sample. The detailed procedure is presented in Algorithm 1 of Appendix C.2. 

We note that in the subsequent phase of manifold learning through SDAE, these calculated data Shapley values for negative samples play a crucial role in the computation of the isotropy constraint loss term, as described in Equation (6). This constraint ensures uniform changes in data Shapley values across all orientations. Hence, our proposed manifold learning with structure preservation and isotropy constraint renders the constructed negative sample Shapley field more suitable for subsequent cohort discovery, as exemplified in the transition from Figure 1(b) to Figure 1(c).

Reviewer comment (Novelty):
What are the broader implications of the proposed method and its conclusions? Can they offer insights for future modeling or labeling processes in ML/DL research?

Author response:
The novelty of our proposed cohort discovery approach stems from its innovative extension of the data Shapley value concept to enable a Shapley-based exploration of interrelationships between samples. This extension moves beyond traditional feature-based similarity methods, asserting that valuable cohorts should exhibit similar distributions with high data Shapley values.

Let us illustrate with a practical example: when a patient exhibits symptoms of upper respiratory distress, outpatient physicians typically prioritize measuring the patient's body temperature and inquiring about any recent travel history or epidemiological contacts. This essentially constitutes leveraging an identified negative sample cohort for efficient medical delivery, specifically for the diagnosis of high-risk infectious viral diseases such as COVID. This is crucial because, in this scenario with limited medical resources, it is impractical to triage all coughing patients into isolation wards.

In this routine example, the "patient without fever" can be considered to exhibit a high data Shapley value in the context of the COVID task, representing a high-temperature zone within the proposed negative Shapley field. Although our proposed approach stems from clinical experience and the examination of relevant medical data, we believe that this innovative approach offers valuable insights for other application domains (e.g., to discover typical classification and exclusion strategies in complex problems). Its recognition aids in deepening the understanding of negative samples, particularly in discerning the typical patterns of negative samples specific to certain tasks.

Reviewer comment (Reproducibility):
How is the DAE stacked? Are the reconstruction loss and iso loss applied to each layer of the stacked DAE?

Author response:
In the process of stacking multiple DAEs, we adhere to the standard practice introduced in the original SDAE paper by Vincent et al. (2010). The model architecture of SDAE-based manifold learning is presented in Figure 6 of Appendix D.1, where both the reconstruction loss and the isotropy loss are applied to each individual DAE of SDAE.

To provide a more detailed view, Figure 6(a) illustrates the computational details of the $k$-th DAE, where both the reconstruction loss and the isotropy loss are employed in this process. Both losses collectively contribute to the training of robust feature extractors, leveraging clean and corrupted inputs simultaneously. After training the encoder of the $k$-th DAE, it is applied to clean inputs for transformation, as depicted in Figure 6 (b). This iterative training process continues, as shown in Figure 6 (c), until the last DAE generates the final encoded representation within the manifold space. This encoded representation serves as the foundation for subsequent cohort discovery.

Reviewer comment (Experiments):
While the proposed approach is reasonable, the authors do not offer any alternatives or baselines. One that immediately comes to mind is influence-functions, which can offer much the same information as Shapley values for this application.

Author response:
**Weakness 1**

We would like to clarify that previously, we compared our proposed approach against several baseline methods. These baselines included contrastive principal component analysis and positive-unlabelled learning methods, and the corresponding results were discussed in Appendices G.2 and G.3, respectively.

The central concept of our proposed approach revolves around incorporating data valuation into spatial clustering analysis among negative samples, through a Shapley-based exploration of interrelationships between the samples. We choose to utilize data Shapley values for data valuation in this process, given that data Shapley values are recognized as a popular and well-established equitable data valuation technique in recent years, offering distinct advantages over other data valuation techniques examined in related work (Ghorbani \& Zou, 2019; Rozemberczki et al., 2022).

We appreciate the reviewer's suggestion of including influence functions [a] in our comparison. Influence functions also serve as a data valuation technique, which assesses how the prediction model changes when the weight of a single sample is slightly altered. To provide a more comprehensive experimental evaluation, we have incorporated influence functions as a baseline in our study.

Specifically, in line with standard practice for comparing with influence functions, we employ the leave-one-out (LOO) method (Ghorbani \& Zou, 2019; Rozemberczki et al., 2022). We would like to note that the comparison between data Shapley values and influence functions in terms of data valuation is also evaluated in Ghorbani \& Zou (2019), demonstrating a significant performance advantage for data Shapley values over influence functions. 

Our evaluation results for influence functions are detailed in Appendix G.4. Specifically, we present the cohort discovery results of influence function-based data valuation for AKI prediction in Figure 12. Figure 12(a) displays the histogram of influence function values among all negative samples.
The distribution appears as a single normal distribution with a mean of zero, which is different from the distribution of data Shapley presented in Figure 2(a).
Figure 12(b) illustrates the distribution of all negative samples in terms of their influence function values in the manifold space.
From this figure, we cannot discern significant high-temperature areas, as samples with different temperatures are completely mixed together.
This suggests that if influence functions are used for data valuation, they do not exhibit a clear trend of spatial proximity similarity in the feature vector space, which may make it challenging to effectively benefit spatial clustering algorithms for further cohort discovery.
This concern is further experimentally validated, as shown in Figure 12(c). We implement the same subsequent clustering steps as in Figure 2(c) in the negative sample space based on influence functions (i.e., selecting 40\% of high influence function values as input for DBSCAN).
From Figure 12(c), it can be seen that DBSCAN fails to generate meaningful clustering results and only produces a single cluster. 
These cohort discovery results indicate that influence function-based data valuation does not reveal meaningful medical cohorts for AKI prediction.

Next, we compare the AKI prediction performance achieved by a new setting, "$d_i^-$ with $IF>0$," which includes only the negative samples with positive influence function values, with three different settings of our proposal. Figure 13 presents the experimental results in terms of AUC (mean $\pm$ std) from five repeats. It is observed that influence functions underperform data Shapley values, i.e., "$d_i^-$ with $IF>0$" vs. "$d_i^-$ with $s_i>0$", and generally achieve lower AUC than using all negative samples (the setting "All $d_i^-$"). This indicates that the setting ``$d_i^-$ with $IF>0$'' deletes informative data samples from all negative samples rather than retaining them.
Consequently, this set of controlled experiments confirms that, compared to influence functions, data Shapley values are a more suitable and effective data valuation measure for the discovery of the relevant medical cohort in AKI that we focus on.

Reviewer comment (Experiments):
The insights generated by the proposed approach are not dissimilar to what one might see using feature-importance analysis of other types. Can these be compared with the proposed method?

Author response:
**Weakness 2**

Regarding this comment that mentions "the insights generated by the proposed approach," we think that these insights should pertain to the findings presented in Figure 3 of the submitted paper. We would like to elucidate that the lab test patterns showcased in Figure 3 offer a comparative view between the cohort of interest against all the other samples outside the cohort, in terms of the lab test value probability density among the involved samples. Through this analysis, we aim to unveil the distinctive characteristics of each discovered cohort, to further investigate its clinical significance, which has been validated by clinicians. It is essential to emphasize that the lab test patterns displayed in Figure 3 pertain to samples within a specific cohort compared to all the other samples. Consequently, various dominant features emerge within different cohorts.

Feature-importance analysis, on the other hand, typically measures the importance of each feature in predicting the output of the model, from our understanding. In other words, the feature-importance analysis can obtain the distribution of features based on labels. However, for our application scenario, the process is reversed, meaning we obtain cohorts based on the similarity of feature distributions by constructing a negative sample Shapley field. From this perspective, these two types of work are difficult to compare with each other because they are tools designed to address different problems. We note that in the medical context which we are focusing on, cohorts do not have objective labels. We hope that the explanation above clarifies your concern, and if you still expect specific comparative experiments, we would greatly appreciate more specific details on this matter.