Interpretability-driven active feature acquisition in learning systems

Authors: Osman Berke Guney, Ketan Suhaas Saichandran, Karim Elzokm, Ziming Zhang, Vijaya B Kolachalama
Primary area: interpretability and explainable AI
Keywords: Active feature acquisition, Model interpretability

Abstract:
In real-world applications like medicine, machine learning models must often work with a limited number of features due to the high cost and time required to acquire all relevant data. While several static feature selection methods exist, they are suboptimal due to their inability to adapt to varying feature importance across different instances. A more flexible approach is active feature acquisition (AFA), which dynamically selects features based on their relevance for each individual case. Here, we introduce an AFA framework that leverages SHapley Additive exPlanations (SHAP) to generate instance-specific feature importance rankings. By reframing the AFA problem as a feature prediction task, we propose a policy network based on a decision transformer architecture, trained to predict the next most informative feature based on SHAP values. This method allows us to sequentially acquire features in order of their predictive significance, resulting in more efficient feature selection and acquisition. Extensive experiments across multiple datasets show that our approach achieves superior performance compared to current state-of-the-art AFA techniques, both in terms of predictive accuracy and feature acquisition efficiency. These results demonstrate the potential of explainability-driven AFA for applications where feature acquisition cost is a critical consideration.

Decision: reject
Reviewer scores: 3, 3, 6, 3

Reviews and discussion:
<thread>
creation date:
1730358381857

thread_type:
official_review

<thread_object>
creator: reviewer_9rqa

<content>
summary:
This paper introduces a new approach to active feature acquisition (AFA) that uses SHAP to determine which feature to acquire next given instance. The key idea is to use pre-specified SHAP values on the training set obtained using a pretrained classifier, and then use these values to train a policy network to predict the next most informative feature based on these SHAP values. The authors show that their approach outperforms current state-of-the-art AFA techniques on multiple datasets (4 image datasets and 1 tabular dataset).

strengths:
-	The paper is in general well-written and easy to follow.
-	The authors provide extensive quantitative analysis by thoroughly comparing the proposed method to a wide range of feature selection methods.
-	The two-stage training approach and the use of SHAP values to guide feature acquisition sounds like a promising direction.
-	The proposed method provides state-of-the-art performance.

weaknesses:
-	The authors don't fully justify why they use SHAP values to guide feature acquisition. Although their "Oracle" method, which uses the ground-truth SHAP, seems effective, SHAP might not be the best way to choose the most informative features. SHAP measures how much each feature contributes to a change in the model's prediction, but this doesn't necessarily mean those features are the best for making accurate predictions with only a few features. 
-	The approach might not fully consider how the importance of acquiring a feature can change based on what features are already acquired. The ordering of SHAP value is fixed during training and thus the policy network (even though it takes the sequence of observed features as input) will neglect such dependencies, which could be the key to AFA. 
-	The authors motivated the impact of AFA on healthcare and medical applications, but only one out of five datasets is healthcare-related. The other datasets are general image datasets where the artificial partitioning into smaller patches doesn't reflect real-world scenarios. Using real-world tabular medical datasets (like MIMIC or METABRIC) would better demonstrate the method's relevance to its stated application.
-	The authors do not clearly justify why a decision transformer is used as the policy network and the entire sequence of reward is used as input. It's unclear if performance gains are from this specific architecture or from the SHAP guidance.
-	More in-depth case studies would strengthen the paper. Showing how and why feature acquisition order varies across different samples would provide valuable insights.
-	Training Strategy: The details of the training process, especially how the context length ($\ell$) is chosen and how pre-selection is used, could be clearer. Pseudo-code would improve understanding and reproducibility.

questions:
-	Regarding weakness 1, please elaborate the underlying motivation of using the ordering of SHAP value for training the policy network. For instance, why the features that contribute to the model prediction the most should be most discriminative and thus suitable for AFA?
-	The SHAP ordering of features is treated as a fixed ground truth. However, in reality, the order in which features are most informative might shift depending on the information already acquired. For example, if the model already has access to a highly predictive feature, the relative importance of other features might decrease or change. The current approach, by relying on a pre-specified SHAP ordering, could overlook these dynamic interactions between features, potentially leading to suboptimal feature acquisition choices. Could the authors explain why this is not an issue?
-	How does the policy network prevent the selection of an already-chosen feature? I couldn’t identify any explicit architecture design or loss to prevent the re-selection of previously acquired features as the policy network simply provides a softmax output at each instance.
-	To better illustrate the advantages of the proposed SHAP-based AFA method, could the authors provide some case studies? It would be helpful to see examples where traditional AFA methods fail, but the proposed method, guided by SHAP ordering, produces more accurate (discriminative) predictions. Additionally, consider including a subgroup analysis based on similarities in feature acquisition order. This would help to demonstrate the effectiveness of using SHAP values in guiding AFA.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730608824325

thread_type:
official_review

<thread_object>
creator: reviewer_ycgi

<content>
summary:
This paper introduces a new approach to active feature acquisition, particularly for scenarios where gathering all features is expensive or time-consuming, such as in medical applications. They introduce the first AFA framework that leverages SHAP values to determine instance-specific feature importance rankings. They reframe AFA as a feature prediction task rather than a feature exploration problem, using a decision transformer architecture to predict which feature to acquire next based on SHAP values. They propose a novel two-stage training approach: First stage: Training using SHAP value ranking order. Second stage: Training using predicted feature acquisition order to improve real-world performance. Through experiments across multiple datasets they demonstrate their method achieves superior or comparable results to state-of-the-art AFA techniques.

strengths:
-The paper reframes AFA from an exploration problem to a prediction problem and leverages SHAP values for determining instance-wise feature importance in AFA, which is novel and innovative

weaknesses:
- The paper lacks theoretical justification for why predicting SHAP values should lead to optimal feature acquisition
-This method bridges between interpretability methods and feature acquisition, it doesn't provide any interpretability results 
- No evaluation of robustness to: a) Input perturbations, b) Different model architectures beyond those tested, c) Changes in the SHAP value computation approach

questions:
To improve the paper, I would recommend: Adding detailed analysis of policy network prediction patterns and failure modes, providing thorough ablation studies of the training process
</content>
</thread_object>

</thread>
<thread>
creation date:
1730656174231

thread_type:
official_review

<thread_object>
creator: reviewer_jvzd

<content>
summary:
The goal of this paper is to sequentially acquire features, for applications where using all features is costly. To do that, the paper first uses SHAp to generate feature rankings for all training data, and then train a policy network to predict the next most important feature.

strengths:
The problem is interesting and important. The performance of the proposed method is good according to the experiments.

weaknesses:
There are two main issues with the paper.

1. The main motivation for the method is that acquiring all features can be expensive, time-consuming and sometimes have to be done sequentially. This argument convinced me at the introduction when the authors mention medical settings. However, all experiments were done using image data, except one using email spam data. I cannot think of any realistic settings where one can only query one patch at a time from an image, or one feature at a time from emails. The application of these methods to these datasets are not convincing at all. Since the authors mentioned medical setting, why not using medical datasets to prove the value of the method?

2. Another concern is the use of SHAP to determine the feature importance. For this specific setting, what the authors really need to predict is the feature that will has a big impact on the predictive performance of a model, not features that will make biggest contribution to the output. For example, one feature could make biggest contribution, but if it makes almost the same contribution to every instance, then it will have little impact on the predictive performance because knowing that feature does not increase the accuracy much. I think a more appropriate feature importance should be something like mean decrease in accuracy.

questions:
Did you try using other importance metrics like mean decrease in accuracy? Why did you use SHAP?

Can you find more convincing and realistic applications of your method?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730706610665

thread_type:
official_review

<thread_object>
creator: reviewer_grqh

<content>
summary:
This paper proposes an active feature acquisition (AFA) framework leveraging Shapley Additive Explanations (SHAP) values to determine feature importance on an instance-by-instance basis. The authors train a decision transformer model to imitate a SHAP-based oracle, intending to optimize the sequential selection of features to maximize predictive accuracy while minimizing acquisition cost. The model's training follows a teacher-forcing-style imitation learning approach, complemented by an on-policy adaptation stage. Experiments are conducted on various image datasets to evaluate the method against existing AFA techniques.

### Strengths


### Weaknesses

### Questions

strengths:
- The use of SHAP values for feature ranking supervision in AFA is novel, bringing interpretability-focused methods into the AFA setting.
- The two-stage training strategy seems interesting. It combines imitation learning and on-policy adaptation to aligns feature acquisition with SHAP rankings, improving the model’s robustness.
- The authors provide a farily clear description of the model's structure and training pipeline, which facilitates reproducibility.

weaknesses:
- Reliance on SHAP values for ranking: The effectiveness of SHAP values as a basis for feature acquisition is questionable. Previous works, such as "Marginal Contribution Feature Importance - an Axiomatic Approach for Explaining Data" (Amnon Catav, et al. ICML 2021), have pointed out that SHAP can dilute the importance of redundant features, while "Problems with Shapley-value-based explanations as feature importance measures" (IE Kumar, et al. ICML 2020) highlights how SHAP’s equal distribution of influence may not suit non-additive models. These limitations challenge the suitability of SHAP-based ranking for supervising feature acquisition policies.
- Baseline: The evaluation lacks an essential baseline where features are acquired based on a global feature importance ranking. Specifically, determining a global ranking from the training dataset and using this static ranking to acquire features for each test instance would offer a valuable comparison, assessing the advantage of instance-specific rankings in this AFA context.
- Dataset selection: The experiments focus purely on image classification datasets, which are not the most relevant to AFA. In practical applications like healthcare or finance, where feature acquisition is costly, a dynamic approach to selecting features would be more impactful. Without evaluations on these types of datasets, the real-world utility of the proposed method remains unclear.

questions:
- How might this model perform on datasets more representative of typical AFA applications, such as those in medical diagnostics, where feature acquisition incurs genuine costs? In such scenarios, I believe that acquisition based on global feature ranking is a strong baseline.
- To increase the evaluation's persuasiveness, consider conducting experiments on various tabular datasets.
- How does the model’s SHAP-based feature selection strategy compare with alternative methods, such as conditional mutual information, especially in scenarios with highly correlated features?
</content>
</thread_object>

</thread>
<thread>
creation date:
1732565512781

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Common response

comment:
We acknowledge the valuable suggestions from the reviewers and have made several improvements to the manuscript in response. Specifically, we addressed the following concerns:

1) We expanded our evaluation to include tabular medical datasets.

- We used a dataset based on a study published in Hepatology (https://archive.ics.uci.edu/dataset/878/cirrhosis+patient+survival+prediction+dataset-1), which focused on predicting survival on individual patients with primary biliary cirrhosis (CPS). The dataset contained 312 patients from Mayo Clinic with tabular features, and the goal was to perform a 3-label classification task (0 = D (death), 1 = C (censored), 2 = CL (censored due to liver transplantation)). On this dataset, our model achieved an accuracy of $67.12\\%$ averaged over 8 features. In comparison, the GDFS method [1] achieved an accuracy of $65.57\\%$, and the DIME method [2] achieved an accuracy of $66.91\\%$. 

- We explored another dataset based on a study published in the New England Journal of Medicine (https://archive.ics.uci.edu/dataset/890/aids+clinical+trials+group+study+175), which is AIDS Clinical Trials Group Study (CTGS). The dataset contained healthcare statistics and categorical information about patients who have been diagnosed with AIDS, totaling 23 features. The task is to predict whether or not each patient died within a certain window of time or not, i.e., a binary classification task. For this problem, our model achieved an AUROC of $0.9167$ averaged over 20 features. In comparison, the GDFS method achieved an AUROC of $0.9142$ and the DIME method achieved a score of $0.8947$.   

- We leveraged another dataset that contained detailed health information for 1,659 patients diagnosed with chronic kidney disease (CKD) (https://www.kaggle.com/datasets/rabieelkharoua/chronic-kidney-disease-dataset-analysis). The dataset comprised of 50 features including demographic details, clinical measurements, medication usage, symptoms, quality of life scores, environmental exposures, and health behaviors. The goal was to predict the presence of CKD, which is a binary classification task. Our model achieved an AUROC of $0.8473$ averaged over 20 features, whereas, the GDFS method achieved $0.8060$ and the DIME method achieved $0.7615$.

- Finally, on the Metabric dataset, our model achieved $69.90\\%$ accuracy when averaged over 20 features. In comparison, DIME achieved $71.45\\%$, and the GDFS method achieved $70.11\\%$.

For more info such as performance on these datasets for the Oracle setting, please see the first table below.

2) We incorporated concrete autoencoders (CAE) as a baseline [3], which is a static global feature ranking approach. This comparison emphasizes the advantages of our instance-specific ranking framework over global ranking methods. 
The CAE baseline achieved accuracies of $57.03\\%$ and $59.59\\%$ on the CPS and the Metabric datasets, respectively. Also, the CAE baseline achieved AUROC scores of $0.6018$ and $0.8653$ on the CKD and CTGS datasets, respectively. See above for the comparison with our method.

3) As per the reviewer suggestions, we performed an ablation study to evaluate the effectiveness of using decision transformer versus another architecture, and compared the model results. On CIFAR-10 (CIFAR-100) dataset, we found that our model with decision transformer achieved an accuracy of $78.61\\%$ ($47.06\\%$) averaged over 20 image patches. When the decision transformer was replaced with a ResNet block, the model achieved an accuracy of $76.83\\%$ ($46.70\\%$). On the BloodMNIST dataset, we found that our model with decision transformer achieved an accuracy of $87.93\\%$ averaged over 20 image patches. When the decision transformer was replaced with a CNN block, the model achieved an accuracy of $78.23\\%$. These results demonstrate the advantage of using a decision transformer as the policy network while also highlighting that our method performs reasonably well with other architectures as the policy network.

4) We leveraged LIME to demonstrate the flexibility of our approach in leveraging other feature ranking methods. On the CPS dataset, our model achieved an accuracy of $66.21\\%$ when LIME-based ranking was used. See above for comparison with SHAP-based ranking. Furthermore, we observed that the model performance was relatively unchanged when different SHAP-based feature ranking methods were used. For example, KernelSHAP and TreeSHAP resulted in accuracies of $66.06\\%$ and $65.91\\%$, respectively. For the results on the other datasets, please see the second table below.
</content>
</thread_object>

</thread>


Metareview:
metareview:
This paper proposes a novel active feature acquisition framework that leverages SHAP to generate instance-specific feature importance rankings. By reframing AFA as a feature prediction task, the authors employ a decision transformer-based policy network and a two-stage training strategy (imitation learning followed by on-policy adaptation). The proposed method demonstrates strong empirical performance across several datasets, outperforming state-of-the-art AFA techniques in terms of predictive accuracy and acquisition efficiency.

**Strengths:**
* The use of SHAP values to guide instance-specific feature acquisition is a novel (bridging interpretability and AFA)
* The two-stage training strategy shows promising results.
* Empirical results demonstrate that the proposed method outperforms existing AFA techniques in terms of accuracy and acquisition efficiency.
* The authors provided additional experiments on tabular medical datasets in response to reviewer feedback, addressing concerns about dataset relevance.

**Weaknesses:**
* Reliance on SHAP for feature ranking remains controversial due to its known limitations, particularly with redundant features or non-additive models.
* Most experiments are conducted on image datasets (limiting the real-world applicability of the method)
* It does not dynamically recalculate feature importance, potentially missing dependencies between already acquired and remaining features.
* Lack of theoretical justification for why SHAP-based rankings should lead to optimal feature acquisition

In summary, the paper presents an interesting and novel approach, but concerns about real-world applicability, reliance on SHAP, and lack of theoretical grounding are significant. The additional experiments help strengthen the paper but did not fully resolve the core issues raised.