Learning Causal Alignment for Reliable Disease Diagnosis

## Abstract

Abstract Deep neural networks have demonstrated impressive accuracy in supervised learning tasks. However, their lack of transparency makes it hard for humans to trust their results, especially in safe-critic domains such as healthcare. To address this issue, recent explanation-guided learning approaches proposed to align the gradient-based attention map to image regions annotated by human experts, thereby obtaining an intrinsically human-centered model. However, the attention map these methods are based on may fail to causally attribute the model predictions, thus compromising their validity for alignment. To address this issue, we propose a novel human-centered framework based on counterfactual generation. In particular, we utilize the counterfactual generation’s ability for causal attribution to introduce a novel loss called the C ounter F actual Align ment ( CF-Align ) loss. This loss guarantees that the features attributed by the counterfactual generation for the classifier align with the human annotations. To optimize the proposed loss that entails a counterfactual generation with an implicit function form, we leverage the implicit function theorem for backpropagation. Our method is architecture-agnostic and, therefore can be applied to any neural network. We demonstrate the effectiveness of our method on a lung cancer diagnosis dataset, showcasing faithful alignment to humans.

## 1 Introduction

Current deep learning (DL) models, though have reached remarkable performance, are often biased towards human non-explainable features in the decision-making process Najafabadi et al. (); Geirhos et al. (). Such biases can raise concerns about trustworthiness, impeding these models from being deployed in safety-critic domains such as healthcare. To address this issue, apart from prediction accuracy, it is desirable for DL models to improve their alignment with human decision-making.

###### Example 1.1 .

Take the diagnosis of lung cancer as an example. The standard protocol followed by radiologists is first to identify the region of interest (i.e., nodule), analyze its
attributes (e.g., size, margin), and then assess its risks of developing into cancers Gould et al. (). Nevertheless, this protocol is not adhered to DL models, which primarily rely on data-driven approaches to learn label-related, but unnecessarily explainable features. As shown in Fig. [1](#S1.F1), deep neural networks mainly utilize background features, such as those of the rib and spine, for prediction. Though these nodule-irrelevant features may endow networks with beyond human performance, they are not understandable for radiologists, thus raising severe safety concerns.

To address this issue, many studies attempted to learn human explainable features for prediction. Examples include Liu et al. (); Wang et al. () that incorporated human knowledge with specialized neural network architectures; and Zheng et al. (); Sreedevi et al. () that proposed decision-making systems based on the human cognitive model. In particular, explanation-guided learning approaches Ross et al. (); Ismail et al. (); Gao et al. (); Fei () proposed to align the attention map of DL models to image regions annotated by human experts, thereby enforcing an intrinsically human-centered model. However, the attention map these methods are based on is obtained in a data-driven manner and not necessarily provides a causal attribution for the model’s decision, which largely compromises their validity for alignment.

In this paper, we propose a novel human-centered framework based on counterfactual generation. Specifically, we first leverage the counterfactual generation to identify features that *causally* attribute to the model’s prediction. Then, to align these features with human annotations, we propose a novel CounterFactual Alignment (CF-Align) loss, which punishes the network once the identified causal factors do not align with human annotations. To optimize the proposed loss that involves counterfactual generation utilizing implicit function forms, we propose an implicit gradient solver rooted in the implicit function theorem. Returning to the lung cancer example, Fig. [1](#S1.F1) shows that our method can learn explainable features; as a contrast, features utilized by the baseline DL model are difficult to interpret.

Contributions. To summarize, our contributions are:

- 1.
We propose a counterfactual alignment loss for human-centered AI.
- 2.
We propose to leverage the implicit function theorem for optimization.
- 3.
We achieve more accurate and explainable results than existing methods on real-world data.

Figure: Figure 1: Saliency maps of a baseline DL model Ronneberger et al. () and our method in lung cancer prediction. Lung nodules are marked with red bounding boxes.
Refer to caption: x1.png

###### Example 1.1 .

## 2 Related works

Existing works on explainable AI (XAI) can be classified in various ways Speith (). One such classification distinguishes between human-centered and technology-centered approaches, with the former focusing on the needs of specific end-users and the latter on those of AI professionals. Another classification is based on whether the method is ante-hoc or post-hoc explainable, which respectively refers to instinct explainability and explanation after the model has been trained.

Our method belongs to human-centered, ante-hoc XAI. Therefore, we will provide a detailed discussion of these two branches of works below, leaving the comprehensive review of other XAI methods to Arrieta et al. () and Ali et al. ().

Human-centered XAI aims to develop AI models that behave in a human-understandable manner. To achieve this goal, existing methods have proposed incorporating various human priors, such as knowledge Liu et al. (); Wang et al. (), memory Fu et al. (), and cognitive models (Zheng et al. (); Sreedevi et al. (), into the design of AI models. By doing so, their models could behave in a certain way that is similar to humans, thereby gaining their trust. However, most of these methods require building model-specific architectures, which makes them difficult to adapt to different settings. In contrast, we propose a model-agnostic loss for human-centered learning, which can be easily adapted to various task settings and neural network architectures.

Ante-hoc XAI refers to the design of AI models that are transparent and intrinsically explainable. Examples of such models include white-box models, such as linear regression and decision trees; hybrid models that combine a white-box model with a black-box model for improved performance Nauta et al. (); Wang & Lin (); joint prediction-explanation models that were trained to provide both predictions and explanations Hind et al. (); Rieger et al. (); and methods that achieved explanation through architecture adjustment Zhang et al. (); Chen et al. ().

Of particular relevance to our work are explanation-guided learning approaches Ross et al. (); Ismail et al. (); Gao et al. (); Fei (), which proposed to align model attention or gradient with human-annotated areas. Nevertheless, these attention regions might not align with the causal factors influencing the decision-making process, potentially undermining their appropriateness for alignment purposes. In contrast, our method is based on the counterfactual generation that intrinsically corresponds to causal attributions, which thus ensures the alignment of the decision process with that of domain experts.

Counterfactual explanation. Our work is closely related to the counterfactual explanation methods Verma et al. (); Balasubramanian et al. (); Lang et al. (), which aimed at answering what could the outcome have changed to had input to a model had been changed in a particular way. To implement, they proposed to minimize alterations that changed the prediction. Such a modified region can be taken as the causal factor to determine the model’s prediction Parafita & Vitrià (). Compared to the attention-based methods, this method can causally attribute the model’s decision Parafita & Vitrià (). However, it is important to note that existing counterfactual explanation methods were designed to explain trained black-box models, while our method aims to design an intrinsically explainable model (*i.e.*, ante-hoc explainability).

## 3 Preliminary

In this section, we introduce the problem setting, followed by an overview of the counterfactual explanation that our method builds upon.

Problem setup & notations. We consider the classification scenario, where the system includes an image $\boldsymbol{x}\in\mathcal{X}\in\mathbb{R}^{p}$ and a label $y\in\mathcal{Y}$ from an expert annotator. In addition to $y$, we assume the annotator also provides an explanation $\boldsymbol{e}$ that explains his/her decision for each image.

In practice, the explanation mainly refers to the region of central interest. For example, in a radiology report, the radiologist often explains his/her decision by annotating disease-related/lesion areas. Motivated by this, we assume that for each sample $(\boldsymbol{x},y)$, the explanation can be formed as the mask $\boldsymbol{r}\in[0,1]^{p}$, where $r_{i}=1$ meaning that the feature $i$ belongs to the region of interest. In this regard, the training data we collect can be denoted as $\{\boldsymbol{x}_{i},y_{i},\boldsymbol{r}_{i}\}_{i=1}^{n}$. With this data, our objective is to learn a classification model $f_{\boldsymbol{\theta}}:\mathcal{X}\mapsto\mathcal{Y}$ that i) predicts $y$ accurately, and ii) makes its prediction based on the region $\boldsymbol{r}$.

### 3.1 Counterfactual explanation

We first provide an overview of the counterfactual explanation method Verma et al. () that our method is based on. By generating a modified image and observing its difference from the original image, this framework intends to answer the following counterfactual question: what could have the model made a different prediction had the input $X$ changed in a particular way?

For this purpose, for the classifier $f_{\boldsymbol{\theta}}$ and each sample $(\boldsymbol{x},y)$, this framework generates a *counterfactual image* $\boldsymbol{x}^{*}$ with respect to the counterfactual class $y^{*}\neq y$:

$$ $\boldsymbol{x}^{*}:=\arg\min_{\boldsymbol{x}^{\prime}}\mathrm{CE}(f_{ \boldsymbol{\theta}}(\boldsymbol{x}^{\prime}),y^{*})+\lambda d(\boldsymbol{x}, \boldsymbol{x}^{\prime}),$ (1) $$

where $\mathrm{CE}$ is the cross-entropy loss, $d(\cdot)$ is a distance measure that constrains the modification to be sparse Verma et al. (), and $\lambda$ is regularization hyperparameter.

With such a modified image $\boldsymbol{x}^{*}$, we can then say the modified region $\mathbbm{1}(|\boldsymbol{x}^{*}-\boldsymbol{x}|)$ to be the explanation of $f_{\boldsymbol{\theta}}$’s prediction, in the sense that if the pixels in this region had taken values from $\boldsymbol{x}^{*}$, then the prediction would have been $y^{*}$. Indeed, this means for a human-centered model, the counterfactual modification should belong to the annotated region of interest. This motivates our *counterfactual alignment loss*, as introduced in the next section.

Counterfactual Explanation (CE) vs Adversarial Attack (AA). These two methods are similar in the optimization form Szegedy et al. (); Wachter et al. (), but are intrinsically different in terms of objectives and distance measures Freiesleben (). Specifically, the CE hopes to identify only decision-relevant features. Therefore, the $d(\boldsymbol{x},\boldsymbol{x}^{*})$ should take into account the sparsity, which can be achieved by $\ell_{0}$ or $\ell_{1}$ norm. On the other hand, the AA hopes to generate samples whose modifications are imperceptible, for which one typically adopts $\ell_{2}$ or $\ell_{\infty}$ norm.

## 4 Method

In this section, we propose to align the decision basis of the model with regions provided by human experts. To this end, we propose a regularity term dubbed as the CounterFactual Alignment loss (CF-Align loss). The goal of this loss is to enforce the network’s decision basis (identified with Eq. ([1](#S3.E1))) to align with the region of interest $\boldsymbol{r}$. In this regard, the classifier will be forced to make predictions based on human-explainable features. The overall pipeline of our method is illustrated in Fig. [2](#S4.F2).

Figure: Figure 2: Overview of our method. The overall objective consists of two parts: the cross-entropy loss $\mathcal{L}_{cls}$ and the CF-Align loss $\mathcal{L}_{align}$. In the forward pass, the counterfactual image $\boldsymbol{x}^{*}(\boldsymbol{\theta})$ is used to compute the $\mathcal{L}_{align}$, where $\boldsymbol{x}^{*}(\boldsymbol{\theta})$ takes $\boldsymbol{x}$ and $\boldsymbol{\theta}$ as inputs. In the backward pass, we compute the gradient of $\mathcal{L}_{align}$ w.r.t. $\theta$ with an implicit gradient solver, then optimize the $\boldsymbol{\theta}$ such that the counterfactual modification is within the human annotations boundary $\boldsymbol{r}$.
Refer to caption: x2.png

Formally speaking, for a prediction model $f_{\boldsymbol{\theta}}$, we define the CF-Align loss as:

$$ $\mathcal{L}_{align}(\boldsymbol{\theta}):=\frac{1}{n}\sum_{i=1}^{n}\left\|(1- \boldsymbol{r}_{i})\odot(\boldsymbol{x}^{*}(\boldsymbol{\theta})-\boldsymbol{x })\right\|_{1},$ (2) $$

where $\boldsymbol{x}^{*}(\boldsymbol{\theta})$ obtained from Eq. [1](#S3.E1) is a function of the model parameter $\boldsymbol{\theta}$, $\boldsymbol{r}_{i}$ is a binary mask that represents the region of interest, and $\odot$ denotes the element-wise Hadamard product between matrices.

By combining with this loss to the cross-entropy loss $\mathcal{L}_{cls}(\boldsymbol{\theta})$, the overall training objective for the prediction model $f_{\boldsymbol{\theta}}$ is:

$$ $\mathcal{L}(\boldsymbol{\theta}):=\mathcal{L}_{cls}(\boldsymbol{\theta})+ \alpha\mathcal{L}_{align}(\boldsymbol{\theta}),$ (3) $$

where the hyper-parameter $\alpha$ is the trade-off between classification accuracy and alignment to experts’ annotation boundary. A higher value of $\alpha$ could potentially lead to improved alignment, but it might also compromise prediction accuracy because the model could leverage other features that may not be understandable to domain knowledge for making predictions. Specifically, it was well established that deep learning methods mainly focused on textual features Geirhos et al. (). Besides, existing works in medical imaging have also shown that microscopic features (*e.g.*, contour, curvature) Wang et al. () or contextual features Liu et al. () can improve the prediction power. Moreover, apart from the lack of explainability, such features may fall outside the realm of causal relationships grasped by domain experts and may pertain to spurious correlations. These correlations could potentially lead to non-robustness when faced with perturbations or out-of-distribution scenarios.

Optimization. To optimize Eq. ([3](#S4.E3)), we need to compute the gradient $\nabla_{\boldsymbol{\theta}}\mathcal{L}_{a}(\boldsymbol{\theta})$, which involves the term $\nabla_{\boldsymbol{\theta}}\boldsymbol{x}^{*}(\boldsymbol{\theta})$. The main challenge lies in that the functional form of $\boldsymbol{x}^{*}(\boldsymbol{\theta})$ is not explicit, making it hard to derive the gradient using chain rules.

To compute this implicit gradient, we employ the Implicit Function Theorem (IFT). The idea is to note the fact that $\nabla_{\boldsymbol{x}}\arg\min_{\boldsymbol{x}}g(\boldsymbol{x},\boldsymbol{
\theta})\equiv\boldsymbol{0}$ for any differential function $g$, which allows us to write $\nabla_{\boldsymbol{\theta}}\boldsymbol{x}^{*}(\boldsymbol{\theta})$ as the product of the inverse Hessian $H_{g}[\boldsymbol{x}]^{-1}$ and the mixed derivative $\nabla_{\boldsymbol{\theta}}(\nabla_{\boldsymbol{x}}g)$. Formally, we have:

###### Theorem 4.1 (Inverse Function Theorem (IFT)) .

Consider two vectors $\boldsymbol{x}\in\mathbb{R}^{p},\boldsymbol{\theta}\in\mathbb{R}^{d}$, and a function $g(\boldsymbol{x},\boldsymbol{\theta}):\mathbb{R}^{p}\times\mathbb{R}^{d}
\mapsto\mathbb{R}$. Let $\boldsymbol{x}^{*}(\boldsymbol{\theta}):=\arg\min_{\boldsymbol{x}}g(
\boldsymbol{x},\boldsymbol{\theta})$. In addition, suppose that the following conditions hold: i) $g$ is differentiable, ii) the $\mathrm{argmin}$ is unique for each $\boldsymbol{\theta}$, and iii) the Hessian matrix $H_{g}[\boldsymbol{x}]$ is invertible.
Then, we have:

$$ $\nabla_{\boldsymbol{\theta}}\boldsymbol{x}^{*}(\boldsymbol{\theta})=-H_{g} \left[\boldsymbol{x}\right]^{-1}\nabla_{\boldsymbol{\theta}}(\nabla_{ \boldsymbol{x}}g)|_{\boldsymbol{x}^{*}(\boldsymbol{\theta}),\boldsymbol{\theta }}.$ $$

###### Remark 4.2 .

For modern neural networks, the inverse Hessian is generally intractable to compute. For this case, to acquire $\nabla_{\boldsymbol{\theta}}\boldsymbol{x}^{*}(\boldsymbol{\theta})$, we can use linear system solvers such as the conjugate gradient solver Hestenes et al. () to solve the linear equation $H_{g}\left[\boldsymbol{x}\right]\nabla_{\boldsymbol{\theta}}\boldsymbol{x}^{*}
(\boldsymbol{\theta})+\nabla_{\boldsymbol{\theta}}(\nabla_{\boldsymbol{x}}g)|_
{\boldsymbol{x}^{*}(\boldsymbol{\theta}),\boldsymbol{\theta}}=\boldsymbol{0}$.

By setting $g(\boldsymbol{x},\boldsymbol{\theta}):=\mathrm{CE}(f_{\boldsymbol{\theta}}(
\boldsymbol{x}),y^{*})$, we can compute the implicit gradient $\nabla_{\boldsymbol{\theta}}\boldsymbol{x}^{*}(\boldsymbol{\theta})$ and thereby the gradient $\nabla_{\boldsymbol{\theta}}\mathcal{L}_{a}$ with the chain rule. Besides, it is important to note that our proposed CF-align loss is model-agnostic. Therefore, it can be easily plugged into the training of various neural networks for human-centered learning.

Extension to attributes $\boldsymbol{a}$. In addition to the class label $y$, Eq. [2](#S4.E2) also applies when the annotated regions $\boldsymbol{r}$ explains its decision to attributes annotations $\boldsymbol{a}$, which will subsequently affect the label $y$. For instance, in the context of lung cancer diagnosis, $\boldsymbol{a}$ pertains to clinical attributes related to nodules, which include factors such as size, margin, spiculation, and more. During the diagnostic process, clinicians commonly annotate the nodule’s region and provide assessments of these attributes, which serve as the basis for their diagnostic decisions.

If these attributes are provided during training, then we can leverage them into our framework. In this case, the overall classifier can be written as $g_{\boldsymbol{\gamma}}\circ f_{\theta}$, where $f_{\boldsymbol{\theta}}:\mathcal{X}\mapsto\mathcal{A}$ and $g_{\boldsymbol{\gamma}}:\mathcal{A}\mapsto\mathcal{Y}$. To determine which attributes to modify, we first employ Zhao et al. () to identify key attributes $\boldsymbol{\bar{a}}$ that have causal influences on $y$, and then generate the counterfactual image by:

$$ $\boldsymbol{x}^{*}(\theta):=\arg\min_{\boldsymbol{x}^{\prime}}\mathrm{CE}(f_{ \boldsymbol{\theta}}(\boldsymbol{x}^{\prime}),\boldsymbol{\bar{a}}^{*})+ \lambda d(\boldsymbol{x},\boldsymbol{x}^{\prime}).$ (4) $$

To train $(\boldsymbol{\theta},\boldsymbol{\gamma})$, we can optimize the objective function Eq. [3](#S4.E3) such that the CE loss is replaced with $\mathcal{L}_{cls}(\boldsymbol{\theta},\boldsymbol{\gamma})$ to account for the classifier $g_{\boldsymbol{\gamma}}$ and that the $\boldsymbol{x}^{*}(\boldsymbol{\theta})$ is generated through Eq. [4](#S4.E4). For a new sample $\boldsymbol{x}$ to predict, we first use $f_{\boldsymbol{\theta(x)}}$ to predict attributes $\boldsymbol{a}$ based on explainable features, then predict $y$ based on attributes $\boldsymbol{a}$.

###### Theorem 4.1 (Inverse Function Theorem (IFT)) .

###### Remark 4.2 .

## 5 Experiment

To demonstrate the practicability of enhancing the explainability within our method, we apply our model to the pulmonary nodule benign/malignant classification. It’s important to clarify that our objective is not to attain state-of-the-art performance on this particular task but rather to leverage it as an illustrative example to emphasize our key point.

### 5.1 Dataset & implementation

We consider the LIDC-IDRI dataset Armato III et al. (), which contains imaging data obtained from clinical thoracic CT scans, as well as fine-grained annotations (nodule bounding boxes, malignancy scores, and attributes information) provided by experienced physicians. Prior to analysis, we preprocess the images by resampling the pixel space to a resolution of $1\times 1\times 1\mathrm{mm}^{3}$ and normalizing the intensity based on a window center of $\mathrm{HU}=-600$ and a window width of $\mathrm{HU}=1600$. In order to ascertain whether our method can learn the same features as those used by physicians, we opt to take the entire 2D slice as input, without explicitly cropping out the nodules. Regarding the classification labels, each sample was scored by physicians, with $3$ or lower meaning benign ($y=0$), while those with a score of $4$ or higher are considered malignant ($y=1$). Besides malignancy labels, each sample is accompanied by six clinical attributes: subtlety, calcification, margin, spiculation, lobulation, and texture. A comprehensive description of these attributes and their influence on the disease label can be found in Appx. [B.1](#A2.SS1). In addition, physicians also provide annotations for the nodule bounding box for each sample, which serves as the foundation for annotating attributes and determining the disease label. We divide the dataset into three subsets: training ($n=731$ nodules), validation ($n=244$ nodules), and test ($n=238$ nodules). To avoid possible label leaking, this partition is based on individual patients.

Figure: Figure 3: Comparison with baselines on the LIDC-IDRI dataset. We use (a) saliency map precision to evaluate model explainability and (b) accuracy to evaluate classification performance.
Refer to caption: x3.png

We implement a seven-layer convolutional neural network (Fig. [7](#A2.F7)) to parameterize $f_{\boldsymbol{\theta}}$. We use the Adam optimizer to train our model, with the learning rate set to $0.001$, batch size set to $128$, and epochs set to $300$. During the training, we iteratively optimize over the classification loss $\mathcal{L}_{cls}$ and the alignment loss $\mathcal{L}_{align}$. To generate the counterfactual image, we adopt the LatentCF method Balasubramanian et al. (), which first encodes the image into the latent space and then modifies the latent code for generation. To estimate the implicit gradient $\nabla_{{\boldsymbol{\theta}}}\mathcal{L}_{align}$, we use the conjugate gradient method implemented in the $\mathrm{TorchOpt}$ package Ren et al. (). We use Grad-Cam Selvaraju et al. () to compute the saliency map for each method. For implementation of compared baselines, we directly load their published codes. To remove the effect of randomness, we repeat all experiments using three different seeds.

### 5.2 Comparison with baselines

We evaluate the classification accuracy and explainability of our method and baselines.

Compared baselines. Firstly, we compare with several ante-hoc explainable AI baselines, namely i) BagNets Brendel & Bethge () that integrated the white-box bag-of-features model with a deep neural network to achieve both explainability and performance; ii) CDEP Rieger et al. () that required the model to produce a prediction as well as an explanation (*i.e.*, multi-tasks learning); iii) ICNN Zhang et al. () that modified the architecture of the neural network to achieve object-centered explainability; iv) RRR Ross et al. () and v) CIDG Chang et al. () that constrains the gradient of the input image to be aligned with annotations from radiologists.

Besides, to achieve a comprehensive comparison, we also include the vi) Vanilla method that optimizes only the classification loss and the vii) Oracle method that takes only features annotated by the physicians (*i.e.*, the cropped-out nodule regions) as input.

Metrics. To evaluate model explainability, we employ the saliency map precision, which is defined as the ratio of the saliency map area within the nodule bounding box to the total area of the saliency map. To evaluate the classification performance, we report the accuracy metric.

Comparison over explainability. We report the saliency map precision of our method and baselines in Fig. [3](#S5.F3) (a). Firstly, we can observe that our method reaches a high saliency precision of $81\%$, which means that the decision basis of our model is well aligned with that of the physicians. Such a promising result can be attributed to the fact that our counterfactual generation method can accurately identify the causes for model decision, and that Thm. [A.1](#A1.Ex5) provides an efficient way to estimate the implicit gradient and makes the alignment loss easy to optimize.

Figure: Figure 4: Visualization of the counterfactual images of six examples. For each example, the left column ($x_{0}$), the middle column ($x^{*}$), and the right column ($x^{*}-x_{0}$) respectively represent the original image, the counterfactual image, and the modified regions. Regions with positive modifications are highlighted in red, while those with negative modifications are marked in blue. The corresponding attribute changes are noted in the title.
Refer to caption: x4.png

Furthermore, we have also noticed that other baseline methods struggle to discern features within nodule bounding boxes when making predictions. This implies that their diagnostic decisions lack explainability for physicians. To comprehend such a result, one should note that BagNets Brendel & Bethge (), ICNN Zhang et al. (), and CDEP Rieger et al. () proposed no explicit constraint to learn explainable features. Therefore, for input with complicated backgrounds such as the thoracic CT, these methods can be easily biased by the contextual features in the image. Meanwhile, though methods such as RRR Ross et al. () and CIDG Chang et al. () explicitly constrained the gradient of the model to human decision areas, the gradient method itself has been found unable to fully represent model decision boundary Jain & Wallace (); Grimsley et al. (), making their identified areas failed to align with ground-truth annotations.

Comparison over classification. We report the classification accuracy in Fig. [3](#S5.F3) (b). As shown, our method significantly outperforms the baselines. This outcome is a product of our method’s ability to harness expert-explainable features that can be consistently applied to test data. In contrast, the baselines may utilize other contextual features for decisions, which are mainly pseudo-correlation and are hard to transfer.

Besides, it is also worth noting that in our results, even the oracle method can only achieve a classification accuracy of $72\%$. In contrast, some work Shen et al. (); Xie et al. (); Wu et al. () on pulmonary nodule classification claimed a classification accuracy over $99\%$. This gap can be mainly explained by the Shen et al. (); Xie et al. (); Wu et al. () elimination of uncertain or challenging samples (those with a score of 3 in the data) from the test dataset in Shen et al. (); Xie et al. (); Wu et al. (). Once again, we would like to emphasize that our experiment is not primarily aimed at surpassing existing state-of-the-art classification methods. Instead, our primary goal is to showcase the model’s explainability and its potential benefits for classification tasks.

Visualization of counterfactual images. The counterfactual images generated are presented in Fig. [4](#S5.F4). For each nodule, the left, middle, and right columns correspond to the original image, the counterfactual image, and the modified regions, respectively. As we can see, the counterfactual modifications generally correspond to one or more attributional features. For instance, the two nodules in the first row were transformed from benign to malignant by introducing spiculation around the nodule. Similarly, the two benign nodules in the 2nd row are classified to be malignant after modifying the original sharp margin to poorly defined ones(^1^11Please see [https://pylidc.github.io/annotation.html](https://pylidc.github.io/annotation.html)). These results demonstrate that our counterfactual generation process can effectively identify fine-grained visual features, thereby enabling the diagnosis decision of our model to align more closely with those made by humans.

### 5.3 Visualization of saliency map

To further evaluate the explainability of our method and the ability of counterfactual generation to localize fine-grained nodule features, we visualize the saliency maps (Fig. [5](#S5.F5)) and the generated counterfactual images (Fig. [4](#S5.F4)).

Figure: Figure 5: Saliency map visualization of our method and compared baselines. Nodules are marked by red bounding boxes. More examples can be found in Fig. [8](#A2.F8).
Refer to caption: x5.png

Visualization of saliency maps. The saliency maps of our method and compared baselines are shown in Fig. [5](#S5.F5). As we can see, our method can accurately identify nodule-related features, while baseline methods mostly focus on areas irrelevant to the nodule, such as the rib, pleural, and spine. These results verify that our method can effectively constrain the neural network to learn human-centered features, making our diagnosis process intrinsically explainable.

### 5.4 Radiology report generation

To present our explanations to human end-users in a clear and accessible manner, we use a Large Language Model (LLM)-based interface to generate a structured report. Specifically, we employ GPT-4 as a backend through prompting Brown et al. (). The prompt we provide to the language model contains three parts, namely a task description, the predictions, and an explanation that is composed of the disease label, attributes, and saliency map from our classifier.

To ensure the report is well-organized and maintains a professional appearance, we present GPT-4 with various standard examples provided by radiologists from Irvin et al. () and instruct it to generate a report that adheres to the format of these examples. Specifically, the prompt we provide to GPT-4 is formated as follows:

- 1.
Task description: You are a machine that explains the decision of a deep learning model to human end-users.
- 2.
Predictions and explanations: [size][attributes] lesion found in [position]. (if malignant) These features may suggest a malignant etiology. (if benign) These features suggest the nodule is stable and highly likely benign. [follow-up schedule suggestions].
- 3.
Canonical examples: [reports by physicians](^2^22Please refer to Appx. [A.3](#A1.SS3) for the examples we use.).

To illustrate the fidelity of the generated reports, we provide two examples in Fig. [6](#S5.F6). In the 1st example, the nodule exhibits malignant features such as poorly defined margin, spiculation, and lobulation, while in the 2nd example, the nodule displays benign features such as solid texture and sharp margin. Our evaluation focuses on the accuracy and comprehensiveness of the generated reports. As shown, the findings section of the report clearly states the position and attributes of the nodules. Based on such findings, the impressions section provides a correct diagnosis and a detailed explanation of the underlying reasons for such a diagnosis. Further, physicians offer suggestions on follow-up examination schedules based on the disease severity. The generated reports summarize these explanations, which can effectively communicate the diagnosis results to human readers.

Figure: Figure 6: Radiology reports generated by GPT-4 that summarize our explanations composed of the saliency map (attached to the original image in the left corner, findings regarding attributes, and the final prediction and suggestions.
Refer to caption: x6.png

## 6 Conclusions

In this paper, we present a counterfactual-based framework aimed at aligning the model with human-explainable features for prediction. To this end, a counterfactual alignment loss is introduced to ensure that the model only modifies regions within human annotations during counterfactual generation. To optimize this loss, we leverage the implicit function theorem to compute the gradient of the alignment loss, which involves implicit forms in the counterfactual generation process. Our framework’s effectiveness is demonstrated by its ability to guide the prediction model in leveraging nodule-related features for the diagnosis of pulmonary nodule malignancy.

## Appendix A Method

### A.1 Preliminary on matrix derivative

For $y=f(\boldsymbol{x})$, where $f:\mathbb{R}^{p}\mapsto\mathbb{R}$ and $x\in\mathbb{R}^{p}$, the gradient vector is defined as:

$$ $\nabla_{x}f:=\left[\frac{\partial f}{\partial x_{1}},\cdots,\frac{\partial f}{ \partial x_{p}}\right]^{T}.$ $$

For $\boldsymbol{x}=g(\boldsymbol{\theta})$, where $g:\mathbb{R}^{m}\mapsto\mathbb{R}^{p}$, $\boldsymbol{\theta}\in\mathbb{R}^{m}$, and $\boldsymbol{x}\in\mathbb{R}^{p}$, the Jacobian matrix is defined as:

$$ $\nabla_{\boldsymbol{\theta}}\boldsymbol{x}:=\begin{bmatrix}\frac{\partial x_{1 }}{\partial\theta_{1}}&\cdots&\frac{\partial x_{1}}{\partial\theta_{m}}\\ \vdots&\ddots&\vdots\\ \frac{\partial x_{p}}{\partial\theta_{1}}&\cdots&\frac{\partial x_{p}}{ \partial\theta_{m}}\end{bmatrix}_{p\times m}.$ $$

For $y=f(\boldsymbol{x})$, the Hessian matrix is defined as the Jacobian of the gradient vector. That is:

$$ $H_{f}[\boldsymbol{x}]:=\nabla_{\boldsymbol{x}}(\nabla_{\boldsymbol{x}f})= \begin{bmatrix}\frac{\partial^{2}f}{\partial x_{1}^{2}}&\cdots&\frac{\partial^ {2}f}{\partial x_{1}\partial x_{p}}\\ \vdots&\ddots&\vdots\\ \frac{\partial^{2}f}{\partial x_{p}\partial x_{1}}&\cdots&\frac{\partial^{2}f} {\partial x_{p}^{2}}\end{bmatrix}_{p\times p}$ $$

### A.2 Proof of the Inverse Function Theorem (IFT)

###### Theorem A.1 (IFT) .

Consider two vectors $\boldsymbol{x}\in\mathbb{R}^{p},\boldsymbol{\theta}\in\mathbb{R}^{m}$, and a function $f(\boldsymbol{x},\boldsymbol{\theta}):\mathbb{R}^{p}\times\mathbb{R}^{m}
\mapsto\mathbb{R}$. Let $\boldsymbol{x}^{*}(\boldsymbol{\theta}):=\arg\min_{\boldsymbol{x}}f(
\boldsymbol{x},\boldsymbol{\theta})$. Suppose that the following regularities hold:

- 1.
$f$ is continuous and differentiable,
- 2.
the $\mathrm{argmin}$ is unique for each $\boldsymbol{\theta}$,
- 3.
the Hessian matrix $H_{f}[x]$ is invertible.

Then, we have:

$$ $\nabla_{\boldsymbol{\theta}}\boldsymbol{x}^{*}(\boldsymbol{\theta})=-H_{f} \left[\boldsymbol{x}\right]^{-1}\cdot\nabla_{\boldsymbol{\theta}}(\nabla_{ \boldsymbol{x}}f)|_{\boldsymbol{x}^{*}(\boldsymbol{\theta}),\boldsymbol{\theta }}.$ $$

###### Proof.

Since the $\boldsymbol{x}^{*}(\boldsymbol{\theta})$ is defined by $\arg\min_{\boldsymbol{x}}f(\boldsymbol{x},\boldsymbol{\theta})$, we have:

$$ $\nabla_{\boldsymbol{x}}f|_{\boldsymbol{x}^{*}(\boldsymbol{\theta}),\boldsymbol {\theta}}=[\boldsymbol{0}]_{p\times 1}.$ $$

Then, we have:

$$ $\nabla_{\boldsymbol{\theta}}(\nabla_{\boldsymbol{x}}f|_{\boldsymbol{x}^{*}( \boldsymbol{\theta}),\boldsymbol{\theta}})=\nabla_{\boldsymbol{\theta}}([ \boldsymbol{0}]_{p\times 1})=[\boldsymbol{0}]_{p\times m}.$ $$

Now, applying the law of total derivation, we have:

$$ $\nabla_{\boldsymbol{\theta}}(\nabla_{\boldsymbol{x}}f|_{\boldsymbol{x}^{*}( \boldsymbol{\theta}),\boldsymbol{\theta}})=\left\{\nabla_{\boldsymbol{x}}( \nabla_{\boldsymbol{x}}f)\cdot\nabla_{\boldsymbol{\theta}}\boldsymbol{x}^{*}+ \nabla_{\boldsymbol{\theta}}(\nabla_{\boldsymbol{x}}f)\right\}|_{\boldsymbol{x }^{*}(\boldsymbol{\theta}),\boldsymbol{\theta}}$ $$

Denote $H_{f}[\boldsymbol{x}]:=\nabla_{\boldsymbol{x}}(\nabla_{\boldsymbol{x}}f)$ as the Hessian matrix. The above two equations together mean:

$$ $\left\{H_{f}[\boldsymbol{x}]\cdot\nabla_{\boldsymbol{\theta}}\boldsymbol{x}^{* }(\boldsymbol{\theta})+\nabla_{\boldsymbol{\theta}}(\nabla_{\boldsymbol{x}}f) \right\}|_{\boldsymbol{x}^{*}(\boldsymbol{\theta}),\boldsymbol{\theta}}=[ \boldsymbol{0}]_{p\times m},$ $$

which shows the statement in the theorem.
∎

###### Theorem A.1 (IFT) .

###### Proof.

### A.3 Prompt examples

## Appendix B Experiment

### B.1 Description of nodule attributes

**Table 1: Attributes of pulmonary nodules**
| Name | Description | How to convert to a binary label |
| --- | --- | --- |
| Subtlety | Size of the nodule | Subtle: score $\leq 4$; Obvious: score $=5$ |
| Calcification | Whether calcification is present | Absent: score $=6$; Present: score $\leq 5$ |
| Margin | Whether the margin is well-defined | Sharp: score $=5$; Pooly-defined: score $\leq 4$ |
| Spiculation | Whether spiculation is present | Absent: score $=1$; Present: score $\geq 2$ |
| Lobulation | Whether lobulation is present | Absent: score $=1$; Present: score $\geq 2$ |
| Texture | Radiographic solidity | Solid: score $=5$; GGO/Mixed: score $\leq 4$ |

### B.2 Network architecture

Figure: Figure 7: Architecture of the convolutional neural network used in the experiment.
Refer to caption: x7.png

### B.3 Extra visualization results

Figure: Figure 8: Visualization of the saliency maps of different methods. Each row indicates an instance.
Refer to caption: x8.png

## Peer Reviews

confidence: 3
contribution: 4
initial_rating: 6
presentation: 3
questions: - Is the loss function in line 194 optimized directly? From Algorithm 1, it appears that this loss function is optimized in a two-step process, i.e. finding $x_i^*$ by minimizing $L_{ce}$ and then finding $\theta$ by minimizing $L_{align}$.
soundness: 3
strengths: - To address the limitations in previous literature, where identified image regions align with expert behaviors only associationally, this paper proposes using counterfactual generation to identify regions that causally determine the model’s decision. The authors show that the generated counterfactual images maximize the probability of causation.
- To align the identified causal factors with radiologist-annotated areas, the paper introduces a causal alignment loss and an algorithm to estimate the gradient of this loss.
- The proposed method shows significant improvement over baseline methods in Class Activation Mapping (CAM) precision and classification accuracy. It effectively distinguishes spurious correlations from true causal factors.
summary: This paper addresses the problem of causal alignment, aiming to recover the causal mechanisms in a decision-making process. It specifically focuses on the classification of medical images, seeking to align the model's reasoning with the causal mechanisms used by experienced radiologists. With access to radiologist-annotated areas, the authors first use counterfactual generation to identify image regions that explain the classification result and then introduce a causal alignment loss to match these identified areas with the radiologist annotations. The alignment loss is optimized using the Implicit Function Theorem and the conjugate gradient algorithm. When attribute descriptions of abnormalities are available, they select the subset of attributes that causally determined the radiologists' labeling for each patient by maximizing the Conditional Counterfactual Causal Effect (CCCE). Finally, they propose a hierarchical alignment process to align both the annotated areas and attribute descriptions with expert annotations.
weaknesses: - The selection of the set of attributes that are causally related to the label requires further justification. The paper claims that the subset $S$ maximizing the CCCE represents the causally related attributes. However, if an attribute $A_k$ is not a cause of $Y$ for some $k \in \{1:p\}$, then we would have $Y_{A_k = 1, A_{-k} = a_{-k}} = Y_{A_k = 0, A_{-k} = a_{-k}}$ for any fixed $a_{-k}$. Thus, the CCCE would be the same for the true causally related subset $S$ and any superset of $S$. It seems that the full set $A$ might also maximize CCCE. However, since only six attributes are annotated in both the LIDC-IDRI and CBIS-DDSM datasets, the proposed algorithm may still work even if $r_i = (1, \dots, 1)^T$ for all $i$.
- The paper claims that previous methods, which regularize the model's input gradient to align with expert-annotated areas (lines 37–40), only capture associational factors. However, the paper later (lines 405–407) attributes the failure of these methods not to their inability to distinguish spurious correlations, but to the limited effectiveness of gradient-based approaches themselves. In fact, even if the identified image regions are only associational factors, regularizing the gradient to expert annotations will rule out spurious correlations. Although the explanation in Figure 1(a) is conceptually reasonable, the simulation does not explicitly demonstrate the advantage of causal factors over associational factors when expert annotations are leveraged.
- It would be helpful to clarify Figure 3, the causal diagram that illustrates radiologists' decision process. It should be emphasized that $A_{1:p}$ represents the experts' "decision" about image attributes, rather than the underlying true attributes of the image. It initially appears to me that the underlying true attributes are inherent features of the image, in which case no causal relationship would exist between $X$ and these true attributes. Besides, how the area marks $m$ fit into the causal diagram could be explained further.
- Could the authors provide the computation time for the proposed method compared to the baseline methods?

---

confidence: 3
contribution: 3
initial_rating: 6
presentation: 4
questions: Please see weakness.
soundness: 3
strengths: This is a good paper, I think. The presentation is concise, and the explanation of theory is clear. Numerical experiments are solid, with detailed description of implementation and comparison of baselines. Also, I think the idea of designing causal alignment loss based on counterfactual generation very fascinating.
summary: This paper proposes a causality-based alignment framework to align the model’s decision with that of experts in disease diagnosis. The key components of the framework are counterfactual generation and causal alignment loss. Counterfactual generation highlights the areas that explain the model’s decision process, and the causal alignment loss minimized the distance between expert-annotated areas/attributes and the generated counterfactual. An optimization algorithm based on Implicit Function Theorem is designed to solve this two-stage problem. The authors then adapted the framework to two scenarios: area of interest annotations and attribute annotations, and develops a hierarchical alignment structure for attribute annotations. Various experiments are conducted and the results show that the causality-based alignment framework is robust against spurious correlation.
weaknesses: My major question or concern for the causality-based alignment framework is computation cost, or scalability. For example, when solving for $x^*$, what if there are multiple counterfactual classes, or even more challenging, $y^*$ is continuous? I guess a straightforward solution would be to bin $y^*$ into binary classes, but that might sacrifice the granularity of counterfactual generation. Also, for hierarchical alignment, conditional counterfactual causal effect score is calculated for each attribute subset. What if the attributes are high-dimensional?

---

confidence: 4
contribution: 4
initial_rating: 5
presentation: 4
questions: Could you clarify the points listed in the Weaknesses section in the text and put the causal model here in the appropriate context?
soundness: 4
strengths: The results are compelling, and the problem it takes up is relevant and timely and promises immediate practical benefits if other researchers pursue it. Further elaboration of the idea shows promise.

The math used makes sense, given the causal model employed. I have some questions about the causal model that can be fixed in the presentation, which I describe below.

I am especially impressed with Figure 4, which shows that the method can infer regions of interest that are closely aligned with those of experts, and with Table 1, which shows the overall accuracy of the method.

In short, the argument that taking causal structure into account in the problem of aligning image analysis with experts in the biomedical sciences is well-put.
summary: The authors present a novel method to help researchers achieve alignment of causal attributions in image analysis for, e.g., lung disease, with that of experts. The novelty is that the alignment is grounded in causal attribution rather than mere association.
weaknesses: The main weakness in the paper, which, as I said, can I think be addressed textually, concerns the causal diagram in Figure 3. This diagram implies that the attributes selected by the procedure are independent of each other and conditional on the existence of a mass. There is no reason I can think of that this must be the case. For example, there could be a latent variable L such that X -> L -> {A1, A2}, in which case conditioning on X would not be sufficient to separate A1 and A2. Also, there's no reason to think that relevant features cannot overlap in the image, at least not in principle, in which case they will not be independent. Some nuance needs to be added to this point. I don't believe the model needs to be changed, but I feel it needs to be presented as a heuristic choice or a choice about the types of feature sets the authors aim to discover.

A second comment is that causal search algorithms are available that, with appropriate background knowledge, could also achieve a result of finding models in the form of Figure 3, and these have not been compared directly for efficacy. I will not recommend such models, as we are not supposed to in these reviews to recommend our own work. Still, the authors could search for these in the literature and explicitly state that perhaps, while they may be effective, a different approach is being taken up and evaluated here, or at least to give reasons why they are not pursued. Some of these methods are highly accurate in terms of the causal graphs that they infer, though the assumptions of these methods could be discussed briefly.

A third comment is that nothing rules out, in principle, the possibility that multiple masses might occur in a single image, and it is not clear what the method would say in cases like this.

A fourth comment is that what model class is being assumed here must be explicitly stated. It's a mixed linear Gaussian/discrete regime, though this should be stated, and if that's not the case, then that should especially be stated up front at the beginning of the article. If it's there and I missed it, then I apologize.

---

confidence: 3
contribution: 3
initial_rating: 6
presentation: 3
questions: Please see Weakness section.
soundness: 3
strengths: High Novelty/Quality: This work develops a novel causal alignment loss and provides an efficient optimization method for solving the learning problem.

High Significance: The empirical results show that their approach not only achieves high diagnostic accuracy but, more importantly, identifies causal factors that contribute to the diagnosis. Accurate identification of causal factors is of paramount importance in the medical field, as it provides informed and precise information for planning subsequent treatments.
summary: This work addresses the problem of identifying medical diagnoses while also uncovering factors that indeed causally contribute to these diagnoses. The authors develop a novel learning and optimization framework that efficiently and accurately achieves this goal. Empirical results demonstrate its superior performance compared to other methods.
weaknesses: Clarity: 

1. In Figure 2 and Figure 1.b, I am confused about the terms "calcification," "margin," and "spiculation." Are these attributes? From my understanding, which was described in the problem setup in section 3  the attributes a_i are binary indicators for the image. Do you mean that each of the binary attributes a_i, for example, indicates the presence of "calcification" in the image? It would be great if the authors could provide a more clear descriptions/ explanations in Figure 2 and Figure 1. b captions.

2. In Equation (2) for counterfactual generation, which distance function are you using? Since the choice of the distance function will indeed affect the downstream causal factor identification and diagnostic accuracy, it would be great if the authors could provide a section for discussion or ablation study to analyze the effect of distance function on the model performance and interpretability. 

Besides common distance metrics (e.g., L1, L2, cosine similarity), possible prior-studied distance functions in the context of counterfactual generation are worthwhile for discussion [https://arxiv.org/pdf/1711.00399]:

Squared Euclidean Distance :
\begin{align} d(x_i,x^c_i) = \sum^d_{k=1} (x_{i,k} - x^c_{i,k})^2 \end{align}

Scaled Squared Euclidean Distance :
\begin{align} d(x_i,x^c_i) = \sum^d_{k=1} \frac{(x_{i,k} - x^c_{i,k})^2}{std_k(x)} \end{align}

Scaled L1 Norm :
\begin{align} d(x_i,x^c_i) = \sum^d_{k=1} \frac{|x_{i,k} - x^c_{i,k}|}{MAP_k} \quad \text{where,} \end{align} \begin{align} {MAP_k} = median_j (|x_{j,k} -median_l(x_{l,k})|) \end{align}

Meta-review: This is a "classical" borderline paper with both strengths and weaknesses. On the positive side, the idea of using counterfactual generation to identify regions that causally determine the model’s decision was considered interesting and novel. 
On the negative side, the most severe concern seems to be the question about the effect of maximizing CCCE on separating the subset of attributes related to the labels. A related question concern the experimentally observed reduction of CCCE after removal of an attribute. Even after the rebuttal and discussion phase, these question could not be answered in a fully convincing way.   After going again over the rebuttal and all discussions, however, I think that this latter issue is probably not a deep conceptual problem, and that over-all, the positive aspects outweigh the negative ones.  Therefore, I recommend acceptance of this paper.

Decision: Accept (Poster)