An X-Ray Is Worth 15 Features: Sparse Autoencoders for Interpretable Radiology Report Generation

## Abstract

Abstract Radiological services are experiencing unprecedented demand, leading to increased interest in automating radiology report generation. Existing Vision-Language Models (VLMs) suffer from hallucinations, lack interpretability, and require expensive fine-tuning. We introduce SAE-Rad, which uses sparse autoencoders (SAEs) to decompose latent representations from a pre-trained vision transformer into human-interpretable features.
Our hybrid architecture combines state-of-the-art SAE advancements, achieving accurate latent reconstructions while maintaining sparsity.
Using an off-the-shelf language model, we distil ground-truth reports into radiological descriptions for each SAE feature, which we then compile into a full report for each image, eliminating the need for fine-tuning large models for this task. To the best of our knowledge, SAE-Rad represents the first instance of using mechanistic interpretability techniques explicitly for a downstream multi-modal reasoning task. On the MIMIC-CXR dataset, SAE-Rad achieves competitive radiology-specific metrics compared to state-of-the-art models while using significantly fewer computational resources for training. Qualitative analysis reveals that SAE-Rad learns meaningful visual concepts and generates reports aligning closely with expert interpretations. Our results suggest that SAEs can enhance multimodal reasoning in healthcare, providing a more interpretable alternative to existing VLMs.

## 1 Introduction

Radiological services are essential to modern clinical practice, with demand rising rapidly. In the UK, the NHS performs over 43 million radiological procedures annually (Lewis et al., ), costing over £2 billion, and demand for scans more than doubled between 2012 and 2019 (NHS England & NHS Improvement, ). A significant portion of these costs addresses rising demand through agency, bank, and overtime staff, but a national imaging strategy notes this funding is unsustainable (NHS England & NHS Improvement, ). Consequently, there’s growing interest in (semi)-automating tasks like radiology report generation, augmentation, and summarization to assist clinicians (Zhu et al., ; Chen et al., ; Pérez-García et al., ), spurred by advances in multimodal text-vision modelling techniques.

Recent architectures that combine vision encoders with pretrained Large Language Models (LLMs) to create multimodal Vision-Language Models (VLMs) have shown impressive performance in visual and language tasks (Liu et al., ; ; Li et al., ; Lin et al., ; Liu et al., ). VLMs have been applied to healthcare tasks, including radiology report generation (Hyland et al., ; Bannur et al., ; Chen et al., ; Stock et al., ; Yang et al., ), typically by mapping image representations into the LLM’s token embedding space. The LLM is fine-tuned to respond to prompts like ‘$<$image tokens$>$ Produce the findings section of a radiology report for this image’ (Pérez-García et al., ). Despite improvements from scaling VLMs (Yang et al., ), hallucinations and disagreements with domain experts remain common (Yildirim et al., ; Jeblick et al., ; Lee et al., ; Tanno et al., ).

Figure: Figure 1: SAE-Rad overview. Panel A: We learn a set of sparsely activating features by training a Sparse Autoencoder (SAE) on class tokens produced by a radiology-image encoder. Panel B: We retrieve the corresponding reference reports for highest activating images for a feature, from which we can produce text descriptions of each feature. Panel C: We pass a new image through the radiology-image encoder and SAE encoder to retrieve the highest activating features. Text descriptions of these features are subsequently used by a pretrained large language model (LLM) to generate a detailed radiology report.
Refer to caption: x1.png

Hallucinations are unavoidable in LLMs (Kalai & Vempala, ; Xu et al., ), and whilst this represents a limitation of current VLM systems designed for radiology report generation, there are other important considerations of using such a system for this task. For current state-of-the-art systems, it is necessary to finetune a multi-billion parameter LLM (as well as projector weights) to perform visual instruction tuning (Liu et al., ), which is computationally intensive and can be prohibitively expensive. Additionally, the generated reports a VLM provides may not be faithful to the underlying computations of the image encoder – we should aim to design a framework which is verifiably faithful to the image model by reverse engineering the computations of the image encoder. This could yield more interpretable results and thus engender more trust in automated radiology reporting systems.

To this end, we introduce SAE-Rad (Fig. 1), a framework which leverages mechanistic interpretability techniques, and in particular sparse autoencoders (SAEs), to directly decompose image class tokens from a pre-trained radiology image encoder (Pérez-García et al., ) into human-interpretable features. These interpretable features are labelled by use of pre-trained LLMs in an automated interpretability pipeline (Bricken et al., ), and can then be compiled into descriptive radiology reports. To the best of our knowledge, this is the first instance of using SAEs explicitly for a downstream task requiring multi-modal reasoning. Our contributions are as follows:
1) Novel SAE: We propose a hybrid architecture which builds on gated SAEs to achieve accurate reconstructions with comparable sparsity to state-of-the-art techniques;
2) Multimodal reasoning pipeline: We develop a framework based on our SAE which leverages automated interpretability to label SAE features using paired text data and ultimately produce radiology reports given an image;
3) Mechanistic interpretability in healthcare: Our case study on healthcare data demonstrates the utility of mechanistic interpretability for downstream tasks in a relatively smaller and significantly more homogeneous dataset than most general-domain text or natural-image datasets.

## 2 Related work

#### Multimodal reasoning

Multimodal reasoning methods like ScienceQA (Lu et al., ) introduced multimodal chain-of-thought (CoT) by zero-shot prompting models to generate rationales and answers simultaneously. Multimodal-CoT (MM-CoT) (Zhang et al., ) extended this with a two-stage framework that separates rationale generation and answer inference using two models of the same architecture. Duty-Distinct CoT (DDCoT) (Zheng et al., ) further factorizes rationales by decomposing the initial question into sub-questions answered by a vision-language model (VLM). Other divide-and-conquer approaches decompose questions into sub-questions but often require training task-specific visual question generation (VQG) models and additional scoring models (Selvaraju et al., ; Uehara et al., ; Wang et al., ). IdealGPT (You et al., ) iteratively decomposes queries and uses a VLM to answer sub-questions, repeating the process if confidence is low. Unlike these methods that rely on decomposing questions or generating rationales through additional models, our approach directly extracts and interprets features from pre-trained image encoders. This enables faithful and transparent multimodal reasoning without the need for extensive fine-tuning or supplementary VQG models.

#### Radiological VLMs

A number of works have finetuned or otherwise trained specialised foundation models for radiological applications including Med-flamingo (Moor et al., ), Med-PaLM M (Tu et al., ), LLava-Med (Li et al., ), Med-Gemini (Yang et al., ), Rad-DINO (Pérez-García et al., ), MAIRA-1 (Hyland et al., ), R2gengpt (Wang et al., ), and Radiology-GPT (Liu et al., ). With regards to radiology report generation, several works have focused on producing both the ‘findings’ and ‘impression’ sections of the reports (Chen et al., ; Jin et al., ; Yan et al., ), whilst others have focussed on the ‘impression’ section (Bannur et al., ), or, most commonly, the ‘findings’ section (Tu et al., ; Miura et al., ; Delbrouck et al., ; Tanida et al., ; Nicolson et al., ). As noted by others (Hyland et al., ; Yu et al., ; Jeong et al., ), studies examining all three settings found that the choice of section(s) to report significantly affects the performance metrics, making comparison between results difficult. For this reason we focus on the most common setting of producing the ‘findings’ section of a radiology report.

#### Mechanistic interpretability

Bricken et al. () demonstrated that SAEs could recover monosemantic features by training on the residual stream of small transformers. The gated SAE was a Pareto improvement over the baseline SAE in terms of sparsity as measured by the L0 and the loss recovered (Rajamanoharan et al., ). There was concern that SAEs would not scale to frontier transformers until recent work by Templeton et al. () which trained SAEs on Claude 3 Sonnet and discovered a large number of monosemantic features. Contemporaneously, SAEs were trained on the class tokens of a CLIP vision transformer (Fry, ), InceptionV1 (Gorton, ), the conditioning embeddings of diffusion models (Daujotas, ), and the vision transformer of a pathology foundation model (Le et al., ). However, in all cases the discovered language/visual features were not used to perform a downstream multimodal task.

## 3 Background

In this section we give a brief overview of mechanistic interpretability, SAEs, and gated SAEs, before introducing our SAE-Rad framework.

### 3.1 Mechanistic interpretability and sparse autoencoders (SAEs)

#### Mechanistic interpretability

Mechanistic interpretability research aims to identify, understand, and verify the algorithms that an ML model implements by reverse engineering a model’s computations into human-interpretable components (Olah et al., ; Rajamanoharan et al., ). Classical approaches attempted to achieve this by analysing the firing patterns of individual neurons, which were interpreted as possible ‘concept representations’. However, this was broadly ineffective as neurons can be polysemantic, meaning that a single neuron may fire on many unrelated concepts (Rajamanoharan et al., ; Bolukbasi et al., ; Elhage et al., ). Polysemantic neurons are believed to arise during training due to the composition of both the linear representation and superposition hypotheses.

#### Linear representation and superposition hypotheses

Motivated by a number of findings which suggest that concept representations are linear (Gurnee et al., ; Olah et al., ; Park et al., ), the linear representation hypothesis states that neural networks represent concepts (sometimes interchangeably referred to as ‘features’) as directions in activation space (Nanda et al., ). This hypothesis can be thought of as being composed of two properties: 1) Linearity: That is, features are represented as directions; and 2) Decomposability: We can understand neural network outputs as a composition of multiple independently understandable features (Elhage et al., ). The superposition hypothesis states that for an intermediate representation of dimension $n$, neural networks will encode $M\gg n$ concepts as linear directions (Rajamanoharan et al., ; Elhage et al., ). These directions form an overcomplete basis of the activation space and must therefore necessarily overlap with each other. However, a single input will only activate a sparse subset of these concepts, leading to minimal interference between the (non-orthogonal) concept directions (Gurnee et al., ; Rajamanoharan et al., ). The superposition hypothesis can be thought of as a form of learned neural network compression and is closely related to compressed sensing. Recent work (Bricken et al., ) has proposed using SAEs to take features out of superposition and learn monosemantic interpretable representations.

#### Sparse autoencoders (SAEs)

SAEs attempt to ‘undo’ superposition by learning the sparse overcomplete basis (Mallat & Zhang, ; Rajamanoharan et al., ) (or dictionary) of the activation space induced by superposition. SAEs attempt to learn both the concept directions and a sparse vector of coefficients for the inputs, that reflect how much each concept is activated for each input (Cunningham et al., ; Bricken et al., ). To align our nomenclature with the recent literature, we will henceforth refer to such sparse vectors of coefficients as ‘feature activations’.

We begin by defining the ‘baseline SAE’ described by Bricken et al. (). Let $n$ be the dimension of the input and output (typically the input is the residual stream of a transformer, and the output is its reconstruction (Elhage et al., )), and $m$ be the SAE hidden layer dimension. Let $s$ be the size of the dataset. Then given encoder weights and biases $W^{\text{enc}}\in\mathbb{R}^{m\times n},\mathbf{b}^{\text{enc}}\in\mathbb{R}^{m}$, and decoder weights and biases $W^{\text{dec}}\in\mathbb{R}^{n\times m},\mathbf{b}^{\text{dec}}\in\mathbb{R}^{n}$, the encoding and decoding operations for a dataset $X\in\mathbb{R}^{s,n}$ are

$$ $\displaystyle\mathbf{h}(\mathbf{x})$ $\displaystyle:=\text{ReLU}(W^{\text{enc}}(\mathbf{x}-\mathbf{b}^{\text{dec}})+ \mathbf{b}^{\text{enc}})$ (1) $\displaystyle\hat{\mathbf{x}}(\mathbf{h}(\mathbf{x}))$ $\displaystyle:=W^{\text{dec}}\mathbf{h}(\mathbf{x})+\mathbf{b}^{\text{dec}}.$ (2) $$

The loss function is then

$$ $\mathcal{L}(\mathbf{x}):=\frac{1}{|X|}\sum_{\mathbf{x}\in X}\left[\|\mathbf{x} -\mathbf{\hat{x}}(\mathbf{h}(\mathbf{x}))\|_{2}^{2}+\lambda\|\mathbf{h}( \mathbf{x})\|_{1}\right],$ (3) $$

where $\lambda$ is an L1 sparsity coefficient. The first term is a reconstruction error measured by the squared distance between the input and its reconstruction, and the second is an L1 regularization loss to induce sparsity. It should be noted that in this regime it is possible to reduce the second term in Eq. 3 by simply decreasing the norm of the encoder weights $W^{\text{enc}}$. It is possible to retain both reconstruction quality and sparsity by a corresponding increase of norm of the decoder weights (Bricken et al., ; Rajamanoharan et al., ). This effect is not desired due to both overflow (decoder norm) and underflow (encoder norm) errors. Additionally, this can cause unstable training when using adaptive optimization algorithms such as Adam (Kingma, )(^1^11This is because the Adam algorithm varies the learning rate according to the reciprocal of the exponential moving average of the mean-square gradient.). To resolve this, the column-wise norm of the decoder $W^{\text{dec}}$ can be constrained during training (Bricken et al., ).

### 3.2 Gated SAEs

As can be seen in Eq. 3, SAEs jointly optimize two opposing objectives: 1) Reconstruction fidelity and 2) L1 regularization as a proxy for sparsity (as measured by L0). This means the SAE is free to trade-off some reconstruction fidelity in order to perform better on the sparsity penalty. One consequence of this is shrinkage (Wright & Sharkey, ). That is, for a fixed decoder, the sparsity penalty pushes the feature activations $\mathbf{h}(x)$ towards zero whilst the squared distance loss encourages $\mathbf{h}(x)$ to be large enough in order to produce high quality reconstructions. Thus, the standard SAE will systematically underestimate the optimal magnitude of feature activations (and simply rescaling these does not necessarily overcome this bias) (Rajamanoharan et al., ; Wright & Sharkey, ).

Rajamanoharan et al. () propose a gated SAE which separates the encoding procedure into two tasks: 1) Detecting which features should activate for a given input (this requires an L1 penalty if the features are to be sparse); and 2) Estimating the magnitude of the feature activations (this does not require an L1 loss; indeed, including this loss here introduces a shrinkage bias). The architecture of the gated encoder is

$$ $\mathbf{h}(\mathbf{x}):=\underbrace{\mathbb{I}[\overbrace{W^{\text{gate}}( \mathbf{x}-\mathbf{b}^{\text{dec}})+\mathbf{b}_{\text{gate}}}^{\bm{\pi}_{\text {gate}}(\mathbf{x})}>0]}_{\mathbf{h}_{\text{gate}}(\mathbf{x})}\odot \underbrace{\text{ReLU}(W^{\text{mag}}(\mathbf{x}-\mathbf{b}^{\text{dec}})+ \mathbf{b}_{\text{mag}})}_{\mathbf{h}_{\text{mag}}(\mathbf{x})},$ (4) $$

where $\mathbb{I}[\bullet>0]$ is an element-wise Heaviside step function and $\odot$ is element-wise multiplication. The $\mathbf{h}_{\text{gate}}$ sub-function learns which features should activate for a given input and $\mathbf{h}_{\text{mag}}$ estimates the magnitude of activations for these features. Here, $\bm{\pi}_{\text{gate}}$ is referred to as the $\mathbf{h}_{\text{gate}}$ sub-function’s ‘pre-activations’. To minimize the number of additional parameters required, $W^{\text{mag}}$ shares the same feature directions as $W^{\text{gate}}$, and is defined as
$W^{\text{mag}}_{i,j}:=\exp(\mathbf{r}^{\text{mag}}_{i})\cdot W^{\text{gate}}_{
i,j}$,
where $\mathbf{r}^{\text{mag}}\in\mathbb{R}^{m}$ is a vector-valued scaling parameter.
Letting $\text{RA}(\cdot):=\text{ReLU}(\bm{\pi}_{\text{gate}}(\cdot))$ denote the rectified pre-activations of the gating sub-function, the loss function is defined as

$$ $\mathcal{L}(\mathbf{x}):=\underbrace{\|\mathbf{x}-\hat{\mathbf{x}}(\mathbf{h}( \mathbf{x}))\|_{2}^{2}}_{\mathcal{L}_{\text{reconstruct}}}+\underbrace{\lambda \|\text{RA}(\mathbf{x})\|_{1}}_{\mathcal{L}_{\text{sparsity}}}+\underbrace{\| \mathbf{x}-\hat{\mathbf{x}}_{\text{frozen}}(\text{RA}(\mathbf{x}))\|_{2}^{2}}_ {\mathcal{L}_{\text{aux}}},$ (5) $$

where $\hat{\mathbf{x}}_{\text{frozen}}$ is a fixed copy of the decoder so that gradients from the auxiliary loss $\mathcal{L}_{\text{aux}}$ do not back-propagate to the decoder weights or bias terms. The auxiliary term $\mathcal{L}_{\text{aux}}$ ensures that $\mathbf{h}_{\text{gate}}$ correctly identifies features necessary for reconstruction, as its (positive) pre-activations must be able to reproduce the input. The sparsity term $\mathcal{L}_{\text{sparsity}}$ applies an L1 penalty to the rectified pre-activations (and thus sparsity is only imposed on the gating sub-function), and the reconstruction term serves the same function as in Eq. 3.

## 4 SAE-Rad

In this section we introduce SAE-Rad. First, we describe the autoencoder architecture, which is based on the gated SAE described in Section 3.2. Then, we describe our end-to-end radiology report generation pipeline.

### 4.1 SAE architecture

It was recently demonstrated that a lower overall SAE loss is achievable without constraining the L2 norm of the decoder weights, or centering the input based on the decoder bias (Conerly et al., ). Concretely, they use the same decoder as in Eq. 2 and define the encoder as

$$ $\displaystyle\mathbf{h}(\mathbf{x})$ $\displaystyle:=\text{ReLU}(W^{\text{enc}}\mathbf{x}+\mathbf{b}^{\text{enc}}).$ (6) $$

Note that the input $\mathbf{x}$ is no longer centered by subtracting the decoder bias $\mathbf{b}^{\text{dec}}$ as in Eq. 1. The sparsity penalty in the loss also now includes the L2 norm of the columns of the decoder $W^{\text{dec}}$:

$$ $\mathcal{L}(\mathbf{x}):=\|\mathbf{x}-\mathbf{\hat{x}}(\mathbf{h}(\mathbf{x})) \|_{2}^{2}+\lambda\sum_{i}\mathbf{h}_{i}(\mathbf{x})\cdot\|W_{\cdot,i}^{\text{ dec}}\|_{2}.$ (7) $$

The feature activation for a feature $i$ is then $\mathbf{h}_{i}(\mathbf{x})\cdot\|W_{\cdot,i}^{\text{dec}}\|_{2}$. The ‘concept directions’ are the unit-normalized decoder vectors $\frac{W_{\cdot,i}^{\text{dec}}}{\|W_{\cdot,i}^{\text{dec}}\|_{2}}$.

The SAE-Rad sparse autoencoder is a hybrid architecture which combines a gated encoder layer with unconstrained decoder norms. Its encoder is defined as

$$ $\displaystyle\mathbf{h}(\mathbf{x})$ $\displaystyle:=\mathbb{I}[W^{\text{gate}}\mathbf{x}+\mathbf{b}^{\text{gate}}>0 ]\odot\text{ReLU}(W^{\text{mag}}\mathbf{x}+\mathbf{b}^{\text{mag}}),$ (8) $$

and the training objective given by

$$ $\mathcal{L}(\mathbf{x}):=\underbrace{\|\mathbf{x}-\hat{\mathbf{x}}(\mathbf{h}( \mathbf{x}))\|_{2}^{2}}_{\mathcal{L}_{\text{reconstruct}}}+\underbrace{\lambda \sum_{i}\text{RA}_{i}(\mathbf{x})\cdot\|W^{\text{dec}}_{\cdot,i}\|_{2}}_{ \mathcal{L}_{\text{sparsity}}}+\underbrace{\|\mathbf{x}-\hat{\mathbf{x}}(\text {RA}(\mathbf{x}))\|_{2}^{2}}_{\mathcal{L}_{\text{aux}}}.$ (9) $$

There are therefore four architectural differences between the gated SAE and SAE-Rad: 1) The L2 norm of the decoder weights are not constrained to unit norm; 2) The L2 norm of the decoder weights are included in the sparsity loss term; 3) The input is not centered by subtracting the decoder bias term; 4) We do not leverage stop gradient operations/decoder copying – this means that we allow gradients to back-propagate to the decoder parameters from the auxiliary loss. SAEs are trained to maximize reconstruction fidelity for a given sparsity level (Rajamanoharan et al., ). We therefore evaluate SAEs based on the L0 norm, which is defined as the average number of firing features for the inputs $\mathbb{E}_{\mathbf{x}\sim X}\|\mathbf{h}(x)\|_{0}$, and the mean-squared error loss as defined in Eq. 3. We demonstrate that this novel SAE architecture outperforms the architecture described by Conerly et al. () on our dataset, achieving both a lower L0 and a lower mean-squared error. Details of this comparison can be found in Section B.1.

### 4.2 SAE-Rad – Automated radiology reporting pipeline

In this section we describe our pipeline to automate the task of radiology report generation. Concretely, a radiographic image $\mathbf{x}$ is passed through a pre-trained and frozen vision encoder $f_{\text{img}}(\cdot):\mathbf{x}\mapsto\mathbf{z}$ to produce an image latent $\mathbf{z}$. We leverage the hybrid SAE architecture described in Section 4.1 to learn feature directions $\frac{W_{\cdot,i}^{\text{dec}}}{\|W_{\cdot,i}^{\text{dec}}\|_{2}}$ and their associated activations $\mathbf{h}(\mathbf{z})\cdot\|W^{\text{dec}}\|_{2}$ from the latents.

In order to generate a text-based report, we produced plain-English descriptions of the learnt sparse dictionary (i.e., a description of what each feature direction represents). To do this we performed automated interpretability (Bricken et al., ) by using a pre-trained and frozen LLM to analyse the ground-truth radiology reports of the highest activating images for each feature. Let $\mathbf{X}_{\text{highest}}^{(i)}$ represent the set of images with the largest feature activations for feature $i$. For each image $\mathbf{x}\in\mathbf{X}_{\text{highest}}^{(i)}$ there is an associated ground-truth radiology report $r(\mathbf{x})$. We collect these reports into a set $\mathcal{R}^{(i)}:=\{r(\mathbf{x})|\mathbf{x}\in\mathbf{X}_{\text{highest}}^{(
i)}\}$.
We then utilized a pre-trained and frozen language model $f_{\text{descriptor}}:\mathcal{R}\mapsto d$ to generate a description $d^{(i)}$ for a feature $i$ by analyzing the set $\mathcal{R}^{(i)}$ as $d^{(i)}=f_{\text{descriptor}}(\mathcal{R}^{(i)})$. This process yielded a set of feature descriptions $\{d^{(i)}\}_{i=1}^{M}$, where $M$ is the total number of features learned by the SAE.

For a new scan $\mathbf{x}$ we identified the set of active features $I(\mathbf{x})$ based on a threshold $\tau$ as

$$ $I(\mathbf{x}):=\{i|\mathbf{h}_{i}(f_{\text{img}}(\mathbf{x}))\cdot\|W^{\text{ dec}}\|_{2}>\tau\}.$ (10) $$

The automated radiology report $R(\mathbf{x})$ is then generated by a pre-trained and frozen LLM from the descriptions of the active features

$$ $R(\mathbf{x})=f_{\text{generator}}(\{d^{(i)}|i\in I(\mathbf{x})\}).$ (11) $$

## 5 Experiments

Our overarching hypotheses are that: 1) SAE features capture meaningful visual concepts even in homogeneous datasets (such as is the case for chest radiographs); 2) The visual concepts captured by an SAE can be appropriately described by a pre-trained LLM by use of automated interpretability techniques with paired text data; 3) Natural language descriptions of visual features in the latent space of an SAE can be composed into high-quality radiology reports without explicit use (or training/finetuning) of a VLM for multimodal reasoning. We assessed these hypotheses with our automated radiology reporting experiment (Section 5.1). We then conducted a number of ablation studies to investigate the effects of model size, different sparsity constraints, and the inclusion of auxiliary information (Section 5.2). Next, we performed a case-study for counterfactual imagine generation to evaluate the learned SAE features (Section 5.3). Finally, we conducted a reader study with a specialist radiologist to assess the quality of our generated reports (Section 5.4).

### 5.1 Automated radiology reporting

#### Dataset description

We trained and evaluated all models on the MIMIC-CXR dataset (Johnson et al., ), a public dataset of 227,835 radiographic studies for a total of 377,110 chest radiographs and associated written text reports. We linked all images to their DICOM metadata files to retrieve scan orientations. We only considered images for which metadata files exist, and retained only antero-posterior(AP)/postero-anterior(PA) scans for training (these are ‘head-on’ scans, as opposed to lateral ones). These are the default views for the generation of diagnostic reports given the increased clarity and ability to visualize the relevant anatomy comprehensively (Hyland et al., ). We then extracted the ‘findings’ section from each text report. The ‘findings’ are a natural language description of all relevant negative and positive features for a given radiograph. Occasionally, the ‘findings’ section is placed into another section with the sub-heading of ‘impression’. Datapoints without either a findings or impression section were discarded. We used the recommended train/test split for MIMIC-CXR, resulting in a total of 239,931 training and 3,403 test images.

#### Evaluation metrics

We evaluated generated radiology reports using both general Natural Language Generation (NLG) metrics (BLEU-4 (Papineni et al., ), ROUGE-L (Lin, ), METEOR (Banerjee & Lavie, )) and radiology-specific metrics (RGER score (Delbrouck et al., ), CheXpert F1 score (Irvin et al., )). While lexical metrics assess n-gram overlap and word order, clinical metrics like RGER and CheXpert F1 attempt to evaluate factual completeness and consistency by analyzing entity-relationship graphs and predicting common chest X-ray pathologies, respectively. NLG metrics can be inadequate for assessing radiology reports as they don’t account for clinical significance (Bannur et al., ), while radiology-specific metrics often rely on specialized models (Yu et al., ) or pre-specified findings classes (Smit et al., ; Bannur et al., ). To address these limitations, the RadFact framework Bannur et al. () uses LLMs to assess sentence-level factuality through bi-directional entailment verification with reference reports— offering a robust evaluation method without relying on pre-specified error types or specialized models. Additional details are provided in Section B.2.

#### Experimental setup

The SAE-Rad framework was trained on class tokens produced by the Rad-DINO vision transformer, using an expansion factor of 64 resulting in a latent dimension of 49,152. The model was optimized using Adam with a learning rate peaking at 5e-5 and a sparsity penalty of 8e-3, trained for 200,000 steps with a batch size of 2048. Claude 3.5 Sonnet was used for automated feature interpretation and report generation, while RadFact evaluation employed Llama3-70B-Instruct. Detailed experimental setup details are given in Section B.3.

**Table 1: Report generation performance on the official MIMIC-CXR test split. BL4 = BLEU-4, RG-L = ROUGE-L, MTR = Meteor. Ma-5 (Macro-F1-5), Ma-14 (Macro-F1-14), Mi-5 (Micro-F1-5), and Mi-14 (Micro-F1-14) represent the clinical CheXbert labeler scores. Bolding represents best performance in the current study or between the upper bound models.**
| Model | RadFact $\uparrow$ | NLG Metrics $\uparrow$ | Clinical Metrics $\uparrow$ |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| F1 | BL4 | RG-L | MTR | RGER | Ma-5 | Ma-14 | Mi-5 | Mi-14 |  |
| Current study |  |  |  |  |  |  |  |  |  |
| Baseline | 30.0 | 3.1 | 18.6 | 23.5 | 14.9 | 38.3 | 25.5 | 45.0 | 42.8 |
| CheXagent | 36.9 | 3.7 | 21.5 | 21.1 | 18.0 | 31.5 | 22.5 | 38.6 | 38.1 |
| SAE-Rad (×64) | 37.2 | 1.9 | 17.1 | 29.1 | 18.2 | 47.2 | 34.3 | 54.4 | 53.2 |
| Upper bound models |  |  |  |  |  |  |  |  |  |
| MAIRA-1 | 47.8 | 14.2 | 28.9 | 33.3 | 29.6 | 47.7 | 38.6 | 56.0 | 55.7 |
| MAIRA-2 | 50.4 | 23.1 | 38.4 | 41.7 | 39.6 | 50.4 | 41.6 | 59.1 | 58.1 |

Figure: Figure 2: SAE-Rad identifies clinically relevant and interpretable features within radiological images. We illustrate a number of pathological and instrumentation features relevant for producing radiology reports. We add annotations (green arrows) to emphasize the presence of each feature.
Refer to caption: x2.png

Figure: Figure 3: SAE-Rad accurately captures features reported by human radiologists and more. Above, we showcase a side-by-side comparison between a ground-truth radiology report and one generated by SAE-Rad. The model successfully identifies key clinically relevant features. SAE-Rad also identifies additional details, such as a right-sided dialysis catheter, without hallucination (we annotate this feature with green arrows for emphasis). SAE-Rad can also miss features when compared to the reference report.
Refer to caption: x3.png

#### Quantitative evaluation

We compared SAE-Rad to the current state-of-the-art radiology reporting systems. CheXagent (Chen et al., ) is an instruction-tuned foundation model for CXRs trained on 1.1M scans for question-answering and text-generation tasks. MAIRA-1 &-2 (Hyland et al., ; Bannur et al., ) are VLMS based on the LLaVA 1.5 architecture (Liu et al., ; ). MAIRA-2 is trained on 510,848 CXRs from four datasets and sets the current state-of-the-art for report generation. The MAIRA systems are not publicly available for result replication, and thus we quote their evaluation values directly as our upper-bound. CheXagent is publicly available, and we therefore performed independent replications for this model for a direct comparison. The ‘baseline’ approach is a naïve method of report generation that uses the report of the closest image in the MIMIC train split. Further details of the ‘baseline’ approach can be found in Section B.6. As Table 1 demonstrates, SAE-Rad underperforms on generic NLG metrics such as BLEU-4. This is expected as we do not try to optimize for any specific ‘writing style’ by fine-tuning an LLM on the reference reports from MIMIC-CXR. Conversely, SAE-Rad demonstrates strong performance on radiology-specific metrics which are clinically relevant, outperforming CheXagent by up to 52% in the CheXpert F1 score (macro-averaged F1-14), and achieving 92.1% and 89.9% of the performance of MAIRA-1 and MAIRA-2 on these scores, respectively.

#### Qualitative investigation

Figure 2 illustrates randomly selected monosemantic visual features from SAE-Rad. As can be seen, the SAE learns human-interpretable visual concepts despite the homogeneity and relatively small size of the dataset. These include dextroscoliosis of the spine (Fig. 2; feature 1), bilateral opacifications (Fig. 2; feature 2), unilateral pleural effusions (Fig. 2; feature 3), and the presence of instrumentation – in this case a pacemaker (Fig. 2; feature 4). In Fig. 3, we illustrate an example ‘findings’ section for a CXR with a number of pathological findings; SAE-Rad is capable of detecting multiple relevant pathologies for a given image. Like other radiology report generation systems, SAE-Rad can miss findings. However, it can also occasionally describe a relevant finding which is otherwise missing from the reference report – an example relating to the presence of a dialysis catheter is shown in Fig. 3.

### 5.2 Ablation studies

We conducted a set of additional experiments which characterize the effects of: 1) Varying the SAE expansion factor; 2) Investigating less sparse (‘dense’) SAEs by reducing the L1 penalty coefficient; and 3) Evaluating the relative benefits of including auxiliary information including the ‘indication’ section of a report as well as previous reports, where available. The experimental setup for these ablations is described in Appendix H.

Figure: Table 2: RadFact performance metrics for different SAE-Rad configurations. /w inds = with indication(s), /w inds + prev.reps = with indications and previous text reports.

As can be seen in Section 5.2, an expansion factor of ×64 produced a higher RadFact F1 score compared with both smaller (×32) and larger (×128) expansion factors. In addition, denser SAEs with a larger L0 norm underperformed sparser models. This suggests that concepts useful for radiology report generation are likely to exist in balance between being too coarse (which may cause ‘feature absorption’— an asymmetric form of feature splitting that can negatively impact the interpretability of an SAE feature (TomasD et al., )), or too fine-grained, as these features may be more difficult to accurately describe given insufficient amount of descriptive detail in the ground-truth reference reports. Section 5.2 demonstrates that the addition of auxiliary information such as the indication, which describes why the patient required the scan in the first instance, can boost the RadFact F1 score, with a large boost to recall. However, in our experiments this caused a small degradation to the precision sub-metric. This finding runs against prior work on the effect of including this section (Yu et al., ; Tu et al., ; Nguyen et al., ) and warrants additional investigation in future. We find that adding both previous indications and prior studies has a net positive effect on the quality of generated reports.

### 5.3 Counterfactual image generation for evaluating features

We evaluated the interpretability and validity of our SAE features using an experiment based on Monteiro et al. ()’s assessment of counterfactual imaging models—focusing on effectiveness, composability, and reversibility. SAE features are interpretable if they correspond to distinct concepts that respond predictably to activation space interventions. We trained a diffusion model conditioned on Rad-DINO class tokens (Pérez-García et al., ) to reconstruct MIMIC-CXR radiographs. During inference, we passed a class token through the SAE, intervened on encoder activations, and reconstructed a “counterfactual” token via the decoder, which conditioned the diffusion model to project interventions into imaging space. We tested whether: 1) interventions alter the reconstructed class token accordingly, 2) changes affect only the targeted feature, and 3) features can be “added” or “removed” by manipulating the same activation. Fig. 4 shows the results for two features (cardiomegaly and pacemaker), demonstrating that our interpretations accurately reflect their impact on model behavior. Further details are given in Appendix D.

### 5.4 Reader study

In a reader study with a specialist radiologist evaluating the quality of the automated radiology reports, 165 sentences from 30 reports (SAE-Rad, CheXagent, and a baseline) were analyzed. SAE-Rad had 7% fewer edits than other models and demonstrated significantly fewer errors with clinical impact, particularly in the “significant” category, where SAE-Rad had almost half the rate compared to others. This highlights SAE-Rad’s potential for radiology report generation in a real clinical scenario. The full study can be found in Appendix E.

Figure: Figure 4: SAE-Rad enables counterfactual image generation and unsupervised segmentation with disentangled class tokens. Row 1 examines a pacemaker, and Row 2 investigates cardiomegaly. Column 1 shows original MIMIC-CXR images, Column 2 shows model reconstructions, and Columns 3 and 4 depict counterfactuals by adding and removing features. The final column demonstrates unsupervised segmentation by comparing counterfactual and original images. Details are in Appendix D.
Refer to caption: x4.png

## 6 Discussion, limitations, and conclusion

In this work, we introduced SAE-Rad, a novel framework that leverages sparse autoencoders to automate radiology report generation. Our approach directly decomposes image class tokens from a pre-trained radiology image encoder into human-interpretable features, which are then compiled into comprehensive radiology reports. The experimental results demonstrate that SAE-Rad achieves competitive performance on radiology-specific metrics, outperforming existing models like CheXagent (Chen et al., ) and approaching the performance of state-of-the-art systems such as MAIRA-2 whilst being trained on a significantly smaller dataset and with a much lower training compute budget; see Appendix F. By reverse-engineering the computations of the image encoder, SAE-Rad provides a framework that is verifiably faithful to the underlying model, enhancing transparency and trustworthiness, which are critical considerations in the healthcare setting.

Qualitative analyses confirm that SAE-Rad successfully captures meaningful visual concepts including the presence or absence of pathological features. These interpretable features contribute to generating detailed and accurate radiology reports, as evidenced by strong performance in the clinical evaluation metrics. Our ablation studies indicate that the choice of expansion factor and inclusion of auxiliary information, such as previous reports and indications, can significantly impact the quality of the generated reports, which is broadly in line with the previous literature on this topic REF.

Our approach has limitations. First, SAE-Rad relies on pre-trained (frozen) models for both the image encoder and the LLM in the interpretability pipeline, potentially introducing inherent biases. However, due to the pipeline’s modular nature, these biases can be mitigated by replacing either model without retraining the SAE if the LLM is swapped. Additionally, SAE-Rad underperforms on general language metrics like BLEU-4, suggesting that while the generated reports are clinically accurate, they may lack the fluency and stylistic nuances of human-generated reports, particularly those in the MIMIC-CXR dataset. Improving these metrics through style-aware radiology report generation (Yan et al., ) is a natural avenue for future work. Overall, SAE-Rad presents a novel and effective approach to radiology report generation by leveraging mechanistic interpretability techniques to extract and utilize human-interpretable features from medical images. By providing a verifiably faithful representation of the underlying computations, SAE-Rad contributes to the development of more interpretable AI systems in healthcare.

## Appendix A Reproducibility Statement

We provide details on the experimental setup, hyperparameters, and data preprocessing steps to ensure the reproducibility of our results. We cannot share the MIMIC-CXR dataset, however it can be accessed through application here: [https://physionet.org/content/mimic-cxr/2.1.0/](https://physionet.org/content/mimic-cxr/2.1.0/).

## Appendix B Automated radiology report experiment

### B.1 SAE architecture comparison

In this section we compare the performance of our novel SAE architecture with the architecture proposed by Conerly et al. (). The following hyperparameters were used to train both SAEs:

- •
Expansion factor of $\times 64$.
- •
Batch size of 2048.
- •
Learning rate of $5\text{\times}{10}^{-5}$.
- •
Linear warm-up of learning rate for the first 1% of training.
- •
Linear warm-down of learning rate for the last 20% of training.
- •
L1 coefficient warmup for the first 5% of training.
- •
Adam optimizer with no weight decay.
- •
Trained for 200,000 optimization steps.

The L1 coefficient was increased for the SAE-Rad architecture in comparison to the Conerly et al. () SAE in order to compensate for the additional auxiliary loss term used to train the SAE-Rad architecture. Table 3 displays the resulting metrics comparing the two SAEs - our proposed SAE attains both a lower L0 and a higher reconstruction accuracy.

**Table 3: Comparison of SAE types.**
|  | L1 coefficient | L0 $\downarrow$ | Explained Variance (%) $\uparrow$ |
| --- | --- | --- | --- |
| Conerly et al. () SAE | $5.3\times 10^{-3}$ | 15.7 | 81.3 |
| SAE-Rad | $8.0\times 10^{-3}$ | 13.6 | 84.3 |

### B.2 Evaluation metrics

#### NLG and classical radiology-specific metrics

We evaluated generated radiology reports using both general NLG metrics and radiology-specific metrics. For lexical evaluation, we report BLEU-4 (Papineni et al., ) for 4-gram overlap based on n-gram precision, ROUGE-L (Lin, ) for longest common subsequence matching, and METEOR (Banerjee & Lavie, ), which performs unigram matching using surface forms, stems, and meanings, computing scores based on precision, recall, and fragmentation assessing word order. Whilst widely reported, lexical metrics do not capture factual completeness or consistency (Miura et al., ; Bannur et al., ) and we therefore also include classical clinical metrics. The RGER score (Delbrouck et al., ), based on the RadGraph model (Jain et al., ), evaluates entity-relationship graphs extracted from reports by matching entities and verifying relationships. Additionally, we report the CheXpert F1 score (Irvin et al., ), utilizing the CheXbert model (Smit et al., ) to predict 14 common pathologies in chest X-rays and calculating the harmonic mean of precision and recall between generated and reference texts. Following recommendations (Miura et al., ; Tu et al., ; Hyland et al., ), we provide micro- and macro-averaged scores over five key observations—atelectasis, cardiomegaly, consolidation, edema, and pleural effusion—as well as the F1-14 score encompassing all observations.

#### RadFact — A robust evaluation framework for radiology

Natural language generation (NLG) metrics are insufficient to appropriately assess radiology report generations as they do not account for the clinical significance of each sentence, and instead treat all words equally (Bannur et al., ). Conversely, radiology-specific metrics are often based on specialised models such as CheXbert (Smit et al., ; Irvin et al., ) or RadGraph (Yu et al., ; Jain et al., ; Delbrouck et al., ), which are themselves limited in that rely on specialised models (Yu et al., ), pre-specified findings classes (Smit et al., ; Bannur et al., ), and/or error types (Chaves et al., ; Wang et al., ). To this end, Bannur et al. () proposed the RadFact framework which leverages LLMs to assess the factuality of each sentence by use of bi-directional entailment verification with the ground-truth reference report. RadFact does not rely on pre-specified error types or radiology-specialized models and is therefore significantly more robust for use to assess the quality of generated radiology reports.

### B.3 SAE-Rad additional experimental setup details

In our instantiation of the SAE-Rad framework (described in Sections 4.1 and 4.2), all bias terms $\mathbf{b}^{\text{enc}},\mathbf{b}^{\text{gate}},\text{ and }\mathbf{b}^{\text
{mag}}$ were initialized to zeros. The elements of the shared encoder weights matrix $W^{\text{gate}}$ were initialized such that all rows point in random directions. The decoder weights $W^{\text{dec}}$ were initialized to $W^{\text{gate}\top}$. For a dataset of size $S$, we trained our SAE on the class tokens $\mathbf{Z}\in\mathbb{R}^{S\times 768}$ produced by the Rad-DINO vision transformer (Pérez-García et al., ), a fine-tuned Dino V2 (Oquab et al., ) on a large dataset of chest X-rays. We use the model weights released at [https://huggingface.co/microsoft/rad-dino](https://huggingface.co/microsoft/rad-dino).
The dataset was shuffled and scaled by a constant such that $\mathbb{E}_{\mathbf{z}\in\mathbf{Z}}[\|\mathbf{z}\|_{2}]=\sqrt{\text{dim}(
\mathbf{z})}$, where $\text{dim}(\mathbf{z})=768$. We used an expansion factor of 64 and thus the latent dimension of our SAE is $49,152$. We used the Adam optimization algorithm (Kingma, ) with $\beta_{1}=0.9$, $\beta_{2}=0.999$, and no weight decay. Our learning rate was increased linearly over the first 1% of training to $5\text{\times}{10}^{-5}$ and then decayed linearly to zero over the final 20% of training. Our sparsity penalty $\lambda$ was linearly increased over the first 5% of training to $8\text{\times}{10}^{-3}$. We trained our model for $200,000$ steps. We used a batch size of 2048. We did not use ghost gradients or resampling strategies as none of our features were dead at the end of training — in other words, all learnt concept directions were activated at least once when passing the dataset through the SAE. We trained all models on a single NVIDIA A6000 GPU. We used Claude 3.5 Sonnet (Anthropic, ) for our automated interpretability pipeline; for a given SAE feature, we retrieved the 10 highest activating images for that feature and passed their ground-truth reference report to Claude with the task of extracting the most consistent information across the reports. The output description was used to label the feature. The full prompts are described in Section B.4.

To generate a report from SAE features, we accrued all activating features and their descriptions, and passed them to Claude for concatenation into a full ‘findings’ paragraph. The full prompts are describe in Section B.5. We evaluate RadFact using Llama3-70B-Instruct.

### B.4 Prompt used to annotate SAE features

In this section we describe the prompt used to instruct Claude 3.5 Sonnet (Anthropic, ) to annotate SAE features. Overall, we instruct Claude to identify commonalities between radiological reports corresponding to the 10 maximally activating examples for a given SAE feature. The prompt encourages the model to use chain-of-thought (Wei et al., ) reasoning in-order to ensure the quality of the feature annotations.

Below is an example output from Claude in response to the above prompt.

Finally, we use regex to parse the description using the asterisk as a separator.

### B.5 Prompt used to compose finding sections via SAE features

In this section we describe the prompt used to instruct Claude to compose the finding sections of a radiology report based on: 1) a patient’s past radiology reports, if they exist, 2) the features that SAE-Rad has identified are present within the image and 3) the indication for the radiograph.
Each feature is associated to an importance score in accordance to the magnitude of the SAE dimension of provenance — the higher the magnitude, the higher the importance score. The scores were normalised to the interval between 0 and 1. Our prompt encourages Claude to focus on features with the highest scores as these were likely to be more significant. See Section C.1 for more example outputs following this prompt, alongside their corresponding radiographs.

### B.6 Baseline experimental setup

In this section, we provide details on the experimental details of our baseline report generation method. In summary, the baseline approach returns the ‘closest’ report in the train split. Explicitly, we encode a given radiograph using Rad-DINO into a single embedding vector $\mathbf{z}\in\mathbb{R}^{768}$ by selecting the class token. Then, we identify the radiograph $y$ with the closest Rad-DINO class token from the MIMIC-CXR train split $X$ as measured by the Frobenius norm $y=\operatorname*{arg\,min}_{p\in X}||\mathbf{z}(x)-\mathbf{z}(p)||_{2}$. We then return the original findings section from $y$ as our predicted, baseline report.

## Appendix C Additional qualitative evaluations

### C.1 Additional radiology report generations

In this section we provide additional qualitative examples of SAE-Rad generated findings sections, highlighting findings jointly described between radiologist generated reports, missing findings by SAE-Rad, and correct findings highlighted by SAE-Rad not present in the original report.
Descriptions and analyses of findings (dis)similarity were produced and reviewed by a clinician.

Figure: Figure 5: Comparison between SAE-Rad Report and the Ground Truth (1). Both reports identify a Swan-Ganz catheter terminating distally in the right pulmonary artery, a left subclavian stent, left as well as right (more pronounced) pleural effusion, and no evidence of pneumothorax. SAE-Rad correctly notes bilateral basal opacities, an asymmetry in lung appearance, consistency in the cardiomediastinal silhouette with respect to previous studies, and a moderate to large right pleural effusion. SAE-Rad misses that the stent also extends to the branchiocephalic vessels, and that that the left effusion is slightly more apparent compared to previous exams.
Refer to caption: x6.png

Figure: Figure 6: Comparison between SAE-Rad Report and the Ground Truth (2). Hyperinflation in the lungs, right lung linear opacities, blunting of the right posterior pleural sulcus and the normal limits of the cardiac sihloutee are noted in both reports. SAE-Rad detects the presence of focal areas of consolidation or atelectasis, the patients known history of emphysema, and the gastric pull-through that remains unchanged from prior exams. SAE-Rad misses the changes of the posterior right ribs and the lack of acute osseous abnormalities.
Refer to caption: x7.png

Figure: Figure 7: Comparison between SAE-Rad Report and the Ground Truth (3). Ground-truth and SAE-Rad reports identify the normal catdiomediastinal contours, clear lung fields, as well as the lack of pleural effusion or pneumothorax. SAE-Rad misses that the patient underwent CABG and that their heart, mediastinum and elevated right hemidiaphraghm remain unchanged. SAE-Rad correctly comments on the lack of focal consolidation and cardiopulmonary abnormalities, and notes acute findings with respect to the patients past diagnosis of melanoma. Additionally, SAE-Rad notes the presence of mediastinal clips and a slight reduction in right lung volume.
Refer to caption: x8.png

Figure: Figure 8: Comparison between SAE-Rad Report and the Ground Truth (4). In this report, SAE-Rad captures nearly all elements from the ground truth. Both reports note hyper inflation in the lungs, no evidence of focal consolidation, pleural effusion, or pneuomothorax. Additionally, both reports note the normal size and shape of the cardiomediastinal silhoutte. SAE-Rad adds the presence of flattented hemiadiapharghms, an enlarged retrosternal air space, observes small pulmonary nodules, and the lack of acute osseous abnormalities. SAE-Rad misses the comparison to the prior patient history, where lungs were previously noted as hyperinflated.
Refer to caption: x9.png

Figure: Figure 9: Comparison between SAE-Rad Report and the Ground Truth (5). The presence of a catheter, as well as small bilateral pleural effusions, an enlarged cardiac silhoutte, and the presences of medium pulmonary vascular congestion with edema is shared across both reports. Both reports also note a lack of pneuomothorax, focal consolidations, and focal airspace opacities. SAE-Rad adds more details on the positioning of the catheter, as well as commenting on the normal shape of the mediastinal contours with no apparent displaced rib fractures. SAE-Rad misses the comparative statements with respect to prior studies in relation to the volume of vascular congestion, the changes in the cardiomediastinal silhouette, and the lack of change of position of the dialysis catheter between studies.
Refer to caption: x10.png

Figure: Figure 10: Comparison between SAE-Rad Report and the Ground Truth (6). In this example, SAE-Rad captures a significant portion of features described in the ground truth report. Both reports note the normal size of the heart, the lack of pleural effusions or acute cardiopulmonary processes, and the stability of appearance with respect to previous exams. SAE-Rad correctly describes the clear appearance of the lungs, as well as the prominence of the mediastinal contours and cardiac silhoutte. Additionally, the lack of severe cardiomegaly, pneumothorax or focal consolidations is noted by SAE-Rad. Comparatively, SAE-Rad fails to note the lack of pneumonia and vascular congestion.
Refer to caption: x11.png

Figure: Figure 11: Comparison between SAE-Rad Report and the Ground Truth (7). SAE-Rad correctly identifies all features in the ground truth. The lack of focal consolidations, pleural effusions and pneumothorax is noted in both reports. Additionally, SAE-Rad describes the normality of the cardiac and mediastinal contours, as well as the stability of the cardiac structures. SAE-Rad correctly identifies reported instrumentation, noting the pacemaker, sternotomy wires in line with the patient’s history, as well as the location and presence of an aortic valve prosthesis. SAE-Rad additionally describes the lack of cardiomegaly or mediastinal widening, the stability of the pulmonary system, and the stability of the image with the respect to previous exams.
Refer to caption: x12.png

### C.2 Image examples of monosemantic features

In this section we showcase highest activating images for a number of features, as well as the corresponding feature explanations.
We highlight the variety of features captured by SAE-Rad, from instrumentation features in Fig. 12, Fig. 13, Fig. 14, as well as visual features such as radiograph inversion in Fig. 16, pathology-related features in Fig. 17, and small details such as piercings in Fig. 18.

Figure: Figure 12: Maximally activating images for a feature corresponding to deep brain stimulators which are typically used to treat Parkinson’s disease.
Refer to caption: extracted/5901539/figures/sae_rad_parkinsons.png

Figure: Figure 13: Maximally activating images for a feature corresponding to orthopaedic rods and screws.
Refer to caption: extracted/5901539/figures/sae_rad_rods.png

Figure: Figure 14: Maximally activating images for a feature corresponding to right sided intra-jugular lines in the presence of sternotomy wires.
Refer to caption: extracted/5901539/figures/sae_rad_right_ij.png

Figure: Figure 15: Maximally activating images for a feature corresponding to female radiographs with no pathology detected.
Refer to caption: extracted/5901539/figures/sae_rad_females_no_pathology.png

Figure: Figure 16: Maximally activating images for a feature corresponding to inverted radiographs.
Refer to caption: extracted/5901539/figures/sae_rad_inverted_cxrs.png

Figure: Figure 17: Maximally activating images for a feature corresponding to bowel obstruction.
Refer to caption: extracted/5901539/figures/sae_rad_bowel_obstruction.png

Figure: Figure 18: Maximally activating images for a feature corresponding to piercings.
Refer to caption: extracted/5901539/figures/sae_rad_tps.png

## Appendix D Counterfactual image generation implementation

#### Model Training

In order to visualise intervention effects, we trained a conditional denoising diffusion probabilistic model (DDPM) Ho et al. () to generate chest radiographs given Rad-DINO tokens.
Our DDPM architecture is a three block, 2D UNet, where each block consists of two residual sub-units with SiLU activation and group normalisation.
The downward path consists of a first block of 2D convolutions, followed by spatial self-attention for the second and third blocks.
The upward path consists of two spatial self-attention blocks followed by upsampling for the last block.
The DDPM is conditioned by linearly projecting the CLS token $\mathbf{z}\in\mathbb{R}^{768}$ to a vector $\mathbf{v}\in\mathbb{R}^{1024}$, broadcasting the result to a 2D tensor $\mathbf{j}\in\mathbb{R}^{128\times 128}$, and concatenating $\mathbf{j}$ channel-wise with a sample $\mathbf{n}\in\mathbb{R}^{128\times 128}$ as an input to the UNet.
The DDPM was trained for 330 epochs using a 90/10 train/val split of the MIMIC-CXR dataset, with a batch size of 256 and an AdamW optimizer with a learning rate of $0.001$.
We follow standard diffusion model training procedures by randomly sampling timesteps $\{t\in\mathbb{Z}|0\leq t\leq 1000\}$, noising an image $\mathbf{x}_{0}\subset\mathbf{X}$ for $t$ steps to obtain noised image $\mathbf{x}_{t}=\sqrt{\alpha_{t}}\mathbf{x}_{0}+\sqrt{1-\alpha_{t}}\epsilon$, where $\alpha_{t}$ is a scalar controlling the variance of $\mathbf{x}_{t}$ as a function of $t$, and $\epsilon\sim\mathcal{N}(\mathbf{0},\mathbf{I})$.
The objective function can be defined over the mean squared error between diffusion model $\epsilon_{\theta}$ outputs conditioned on class tokens $\mathbf{z}$:

$$ $\mathcal{L}=\arg\min_{\theta}\mathbb{E}_{\mathbf{x}_{0}\sim q(\mathbf{x}_{0}), t\sim U(0,1000),\epsilon\sim\mathcal{N}(0,\mathbf{I})}\left[\|\epsilon^{(t)}_{ \theta}(\sqrt{\alpha_{t}}\mathbf{x}_{0}+\sqrt{1-\alpha_{t}},\mathbf{z})- \epsilon\|^{2}_{2}\right]$ (12) $$

Training was implemented using PyTorch (Paszke et al., ) and the diffusers (Huggingface, ) package on a single LambdaLabs 8xNVIDIA H100 cluster.

#### Intervention Experiments

We define a qualitative experiment to assess the visual interpretability of SAE-Rad features by evaluating whether activating or deactivating particular SAE features results in the expected visual changes.
These inteventions can be defined using a constant reassignment operator

$$ $\texttt{do}(i,\beta\ ;\mathbf{h(x)}):\mathbf{h(x)}\mapsto\mathbf{\tilde{h}(x)} _{i,\beta}$ (13) $$

which sets a particular feature in the SAE activation $\mathbf{h(x)}$ at index $i$ to a desired constant value $\beta$.
Following Eq. 9, a given image token $\mathbf{z}$ may be reconstructed into an intervention token $\mathbf{\tilde{z}}_{i,\beta}$ by decoding it’s intervened-upon activation $\mathbf{\tilde{h}(x)}_{i,\beta}$

$$ $\tilde{\mathbf{z}}_{i,\beta}(\mathbf{h}(\mathbf{z})):=W^{\text{dec}}\texttt{do }(i,\beta;\mathbf{h}(\mathbf{z}))+\mathbf{b}^{\text{dec}}.$ (14) $$

Given a feature of interest $i$, intervention quantity $\beta$, and token $\mathbf{z}$, we can compute a single “counterfactual” image $\tilde{I}_{i,\beta}$ by reverse denoising $\tilde{\mathbf{z}}_{i,\beta}+\delta$ using $\hat{\epsilon}_{\theta}$, where $\delta=\mathbf{\hat{z}}-\mathbf{z}$ accounts for the reconstruction error inherent to the SAE between the original token $\mathbf{z}$ and reconstructed $\hat{\mathbf{z}}$.

Using this framework, we assessed the counterfactual cyclic consistency for a given feature by performing two sequential interventions: we generated $\tilde{I}_{i,\beta}$ by activating feature $i$ by $\beta$ for a token $\mathbf{z}$, and subsequently reverted the effects of the intervention by deactivating $\beta$ at $i$ in $\tilde{\mathbf{z}}_{i,\beta}$ to generate $\tilde{I}_{i,0}$.
Intuitively, $|\tilde{I}_{i,0}-\tilde{I}_{i,\beta}|$ should represent the visual concepts of interest.
In our experiments, we empirically set $\beta=15$, and select features 311 (pacemaker feature) and 162 (cardiomegaly) our features of interest.
Features were empirically identified by reviewing highest activating feature subsets for our final SAEs.
Results are showcased in Fig. 4.

## Appendix E Reader study

In addition to qualitative analyses performed in Section 5.1, we performed a reader study to assess radiologist opinions of generated reports compared to currently reported methods.
We followed the reader study design from MAIRA-2 Bannur et al. () - in summary, we presented a radiologist with a radiograph, the indication for the exam, and the generated report, and subsequently tasked the radiologist with assessing reports sentence by sentence, where the radiologist could edit, delete or maintain the sentences.
Where changes were made, we additionally tasked the radiologist ti categorise errors as one or multiple of “omission”, “misclassification”, “overspecific”, “incorrect location”, and “other”.
We omitted the category “incorrect progression”, as we do not include past radiographs for consideration as part of our reader assessment.
Additionally, unlike the MAIRA-2 reader study, errors could be classified as multiple error categories, which allowed for increased granularity in labelling.
Finally, errors were categorised according to their clinical implications - errors could be “significant”, “minor”, or “none”.
The custom user interface (UI) used for the study is showcased in Fig. 19.
For full details on error taxonomy, we refer the reader to Bannur et al. ().

Figure: Figure 19: Custom UI for radiograph evaluation.
Refer to caption: extracted/5901539/figures/sae_rad_eval_ui.png

Overall, a radiologist (ST3) reviewed ten unique radiographs with findings section extracted from the ground-truth closest report baseline described in Section B.6, as well as findings generated by CheXagent and SAE-Rad; in total, thirty individual reports were assessed.
The radiologist was blinded to the model that findings originated from during the study.

### E.1 Results

Overall, a total of 165 sentences were analysed, with an average of 5.5 sentences per report (CheXagent: 6.2, SAE-Rad: 5.9, Baseline: 4.4).
Of 165 sentences, 132 (80%) sentences were edited or deleted, with 33 (20%) sentences maintained as is.
Each report, on average, required 4.4 number of edits.
Split by model, CheXagent totalled 52 (83.8%) edits, the baseline totalled 35 (79.5%) edits, and SAE-Rad totalled 45 (76.3%) edits.

Of sentences that required edits, 81 (61%) sentences overall had no clinical impact, 35 (26.5%) had a minor impact, and 16 (12.1%) had a significant impact.
By model, CheXagent had 31 (59.6%) sentences with edits with no clinical impact, 13 (25.0%) sentences with minor impact and 8 (15.3%) sentences with significant clinical impact, the baseline had 19 (54.3%) sentences with edits with no clinical impact, 11 (31.4%) sentences with minor impact and 5 (14.2%) sentences with significant clinical impact, and SAE-Rad had 31 (68.8%) sentences with edits with no clinical impact, 11 (24.4%) sentences with minor impact and 3 (6.6%) sentences with significant clinical impact.

In total, 105 (53.03%) errors were classified partly as “other”, representing the majority class. 30 (15.15%) errors were omissions, 26 (13.13%) errors were mis-classifications, 25 (12.62%) errors were overspecific, and 12 (6.06%) errors were incorrect location.
Error type and severity distribution are visualised in Fig. 20.
Notably, whilst many “other” type errors were observed, the majority of these errors carried no clinical risk.
In contrast, the “omission” and “misclassification” categories convey much higher risk, with a comparatively higher proportion of minor and significant severity categories.
Overall, for sentences with edits, each sentence was labelled with an average 1.46 $\pm$ 0.67 error categories (SAE-Rad: 1.56 $\pm$ 0.62, CheXagent: 1.35 $\pm$ 0.71, baseline: 1.51 $\pm$ 0.62 error categories per edited sentence).

Figure: Figure 20: Distribution of error types for different models. Notably, SAE-Rad achieves the lowest number of “significant” errors in comparison to alternate methods.
Refer to caption: x13.png

Overall, SAE-Rad requires 7% fewer edits than comparable methods.
Additionally, significant clinical impact error rates are almost half reports generated using our method compared to other models, whilst maintaining comparable minor and no impact error types.
Similarly to results reported by Bannur et al. (), the majority of errors have no clinical impact.
Overall, this showcases the potential our proposed method for radiology report generation in a real clinical scenario.

## Appendix F Compute efficiency comparisons

In this section we provide and discuss the compute estimates for training and inference of SAE-Rad in comparison to both MAIRA 2 and CheXagent. In Table 4 we provide quantitative estimates of the compute efficiency - we estimate that SAE-Rad uses approximately 1000x less train compute, 100x less parameters and 4x less data.
We note that while in this paper we combine the SAE text features into a report using Claude, our method enables the generation of reports using a much smaller LLM, or alternately without an LLM — for example, by concatenating the SAE feature text explanations into a report using only regex. This reduces the inference FLOPs and cost to a negligible quantity. In Table 4, we have estimated the lower bound inference compute requirements for SAE-Rad using this assumption.

**Table 4: Comparison of compute resources for different models.**
|  | SAE-Rad | cheXagent | MAIRA 2 |
| --- | --- | --- | --- |
| Number of unique training images | 239,931 | 1,100,000 | 510,848 |
| Active training parameters | 76M | 8B | 7B |
| Estimated training Tera-FLOPs | 60,000 | NA | 100,000,000 |
| Estimated training cost | $3.25 | NA | $4000 |
| Estimated inference Tera-FLOPs per report | 0.1 | 8 | 7 |
| Estimated inference cost per report | $0.00005 | $0.001 | $0.001 |

## Appendix G Limits of automated quality assessment for radiology reports

As described in Section B.2, NLG metrics such as BLEU-4 (Papineni et al., ), ROUGE-L (Lin, ), and METEOR (Banerjee & Lavie, ) do not account for the clinical relevance of the sentences composing a radiology report. Radiology-specific metrics such as the RGER score (Delbrouck et al., ) and the CheXpert F1 score (Smit et al., ) were consequently developed, seeking to improve upon these limitations by using radiology-specialized models as backbones to compute clinically relevant commonalities between generated and reference reports.
However, CheXpert and RGER rely on pre-specified findings classes and error types (Bannur et al., ), which limits their ability to assess the quality of generated reports when elements outside these specified categories are present. RadFact (Bannur et al., ) was recently proposed to address this limitation; by relying on the logical inference capabilities of large language models (LLMs), RadFact evaluates the correctness and completeness of generated reports outside of pre-specified classes.

RadFact introduces two key metrics: RadFact logical precision and RadFact logical recall, henceforth denoted as RadFact precision and recall.
RadFact precision calculates the proportion of generated sentences that are entailed by the ground-truth report, assessing the truthfulness of the model’s outputs by penalizing hallucinations. RadFact recall measures the proportion of ground-truth sentences that are entailed by the generated report, evaluating the completeness of the generated report by penalizing omissions.

A limitation of RadFact is that it relies on the assumption that the ground-truth report used as a basis for evaluation is a gold-standard label which exhaustively captures clinically relevant detail.
Therefore, in cases where a predicted report correctly includes findings present in the image but omitted from the ground-truth report, RadFact precision will drop. This means that the metric may not fully reflect the model’s ability to generate clinically accurate and comprehensive reports, when the ground-truth reports are incomplete.

We illustrate potential pitfalls of RadFact in Fig. 21 below.
In this case, the ground-truth report fails to report several clinically relevant details observed in the SAE-Rad report, including, for example, the normal size of the cardiac silhouette, and the increased density in the right upper lung field.
We provide additional examples of this phenomenon in Section C.1.

We note that the level of detail for radiographic reports is likely impacted by multiple factors, such as report indication, patient history, urgency of the request, and care setting of the radiograph.
Given we perform evaluation on MIMIC-CXR —a dataset of radiographs captured at an emergency department (Johnson et al., )— SAE-Rad is likely to report clinically correct and relevant statements which are not explicitly mentioned by radiologists.
Therefore, reported metrics must be interpreted with care, and may be underestimates of actual clinical factuality and relevance of generated reports.

Figure: Figure 21: Example image for which an SAE-Rad generated report has significantly more detail than the ground-truth. SAE-Rad captures nearly all features noted in the ground-truth such as: fibronodular changes, a diagnosis of sarcoidosis and pleural effusions. However SAE-Rad also notes the lack of pneuomothorax, pulmonary edema and the increased density in the right upper lung field. Despite the clinical relevance of these elements this report likely scored poorly on RadFact due to the incompleteness of the ground-truth.
Refer to caption: x14.png

## Appendix H Ablation experimental setup

In this section we present the experimental setup for our ablation studies. We evaluated six SAEs on RadFact across three expansion factors and two sparsity levels. We evaluated the SAEs using a set of 60 randomly selected radiographs from the MIMIC-CXR test split. RadFact was evaluated using the “Llama3-70B-Instruct” model.
In this study, we used an earlier prompting template to generate feature explanations and subsequent reports.
Overall, we note that this change resulted in slightly worse quantitative metrics than those that were obtained using the final prompting strategy.

All SAEs were trained for 200,000 steps with a batch size of 2048. Models were trained using an Adam optimizer with no weight decay and a learning rate of $5\text{\times}{10}^{-5}$, as well as a linear warm-up of the learning rate for the first 1% of training, linear warm-down of learning rate for the last 20% of training, and an L1 coefficient warm-up for the first 5% of training.

We summarise experimental differences between runs in Table 5, as well as resultant L0 and explained variance metrics. The L0 and explained variance metrics in Table 5 provide insights into the quality of our SAEs. Notably, the “dense” variants exhibit higher L0 values, indicating reduced sparsity, but achieve greater explained variance. The different variants therefore represent different points on the Pareto frontier between sparsity and reconstruction quality.

**Table 5: Varying hyperparameters and resulting metrics for the six SAEs.**
|  | Expansion factor | L1 coefficient | L0 $\downarrow$ | Explained variance (%) $\uparrow$ |
| --- | --- | --- | --- | --- |
| ×128 | $128$ | $6\text{\times}{10}^{-3}$ | 12.0 | 90.9 |
| ×64 | $64$ | $8\text{\times}{10}^{-3}$ | 13.6 | 84.3 |
| ×32 | $32$ | $9\text{\times}{10}^{-3}$ | 15.1 | 79.2 |
| ×128_dense | $128$ | $3.75\text{\times}{10}^{-3}$ | 26.0 | 93.2 |
| ×64_dense | $64$ | $5\text{\times}{10}^{-3}$ | 28.4 | 88.5 |
| ×32_dense | $32$ | $5.63\text{\times}{10}^{-3}$ | 31.8 | 84.7 |

## Peer Reviews

confidence: 4
contribution: 2
initial_rating: 6
presentation: 3
questions: The proposed report generation method of the paper has some novelty in SAE and activated feature interpretability, and the result also shows leading generation performance. But some important differences of the method are not fully explained and evaluated, and some common issues in report generation such as co-occurence and latent feature clustering are not discussed, which weaken the insight of the paper. In conclusion, I think the paper is on the boarderline so I currently rate the paper as "borderline accept". I may adjust my rating based on the authors' reponse during discussion period.
soundness: 2
strengths: ## Strengths
- The public dataset, codes, and implementation details are provided for reproducibility. 
- Method novelty of some improvements on gated-SAE, which achieves better sparcity and reconstruction quality. 
- Enhancing the model interpretability by SAE decomposing image latent features into radiological descriptions. 
- Comprehensive evaluation metrices and leading performance in MIMIC-CXR Chest X-ray report generation.
summary: The paper proposes a novel radiology report generation framework (SAE-Rad) based on Sparse-Autoencoder (SAE). It decomposes latent image features into human-interpretable features, distils these features into radiological descriptions and compiles them into full report. The experiment results on MIMIC-CXR show the leading report generation performance and efficient training computation.
weaknesses: ## Weaknesses and Questions
- The motivation of the method is still not well explained. In 4.1, the authors mention four differences between the gated SAE and SAE-Rad, but I do not find enough explanation about what is the advantage of these differences and why these differences are important for image-to-text generation. Furthermore, I would like to know whether the entire SAE-Rad framework enhances the general image-to-text generation tasks, or it is specifically designed for radiology report generation? If SAE-Rad is mainly designed for report generation, then the authors should also explain the specific advantages of the four differences in SAE-Rad and the sparse feature interpretability method in radiology report generation task. 
- No ablation study about the improved SAE loss (Eq. 9). Since the authors make some changes in SAE loss, as described in Eq. 9 and the four differences, there should be some ablation studies showing the effectiveness of this novel SAE loss, compared with other losses used in SAE training. 
- There is no discussion for the co-occurence issue in medical report generation. In medical images, multiple diseases or visual markers sometimes have high correlation and frequently shown up together in one image, so this may cause some false findings in the generated reports. Since the proposed SAE-Rad report generation model is based on image latent feature decomposition and activated feature descriptions, so it's important to understand if the feature representations of co-occurence visual markers are fully decomposed and disentangled before generating any feature interpretation descriptions from LLMs. Please provide discussions about the co-occurence issue and the potential limitations of it. 
- The number of comparison methods is not enough for a fully demonstration of the effectiveness of the proposed method. In Table 1, only CheXagent is compared in *Current study*, and MAIRA as upper bound. However, since radiology report generation using MIMIC-CXR has been widely explored in recent years, there are much more methods that could be used for comparison. I think the authors should provide more comparison results with some other recent leading methods for a stronger evaluation. 
- The authors use RadFact as one of the evaluation metrics since it is robust without relying on pre-specified error types or specialized models. But in Table 1, SAE-Rad only gains 0.3 in RadFact compared to CheXagent, which is a marginal improvement, and also a large gap to the upper bound; in the same time, other clinical metrics show large increases. So how to explain this discrepancy in these metrics? Does it mean RadFact score is not sensitive enough? 
- The proposed method is based on the sparse image features (feature directions and associated activations) extracted from SAE-Rad, but the clustering and distributions of the extracted features of all chest diseases/markers are not visualized or discussed. The sparse feature distribution in latent space is important to assess the feature clustering and latent space training, where feature directions of similar visual markers should be clustered closely while different visual markers should push apart their feature directions. I recommand the authors visualize the sparse feature representations of each diseases/markers using T-SNE demo to illustrate the sparse feature clustering.

---

confidence: 5
contribution: 3
initial_rating: 5
presentation: 3
questions: Not sure why the authors did not other additional datasets for independent validation?
soundness: 3
strengths: The computational framework seems reasonable. Each component is pretty well established.
summary: This work combined a SAE vision encoder, an LLM and the linked highest activating features, to generate radiology report. The authors used the public MIMIC-CXR dataset for this work.
weaknesses: However, the originality of the methods is very marginal. The algorithmic or methodological innovation is low.

---

confidence: 3
contribution: 3
initial_rating: 6
presentation: 3
questions: •	Eq: 4 Is there an empirical technique for understanding what an optimal number for /pi_gate should be compared to /h_hate?
•	Difference between SAE and SAE-Rad #2: What would the impact of enforcing decoder sparsity be on the encoder itself? One could likely posit that it could further encourage encoder sparsity?
•	Difference between SAE and SAE-Rad #4: The choice of the L0 norm is seemingly heuristic? Could this be relaxed to have more firing features to better represent entangled concepts? I also wonder if there is something to be gained from exploring the relationships between this optimization paradigm and beta-vaes that expicitly force some level of disentanglement of feature sets.
•	How is set R(i) chosen when creating natural language representations of firing features? I suspect this would substantially depend on the size of R? Moreover, i wonder how representations learned from your SAE vary compared to the usual VAE-style methods. Are SAE grouped images also grouped with traditional VAEs?
•	Figure 2 is very interesting. Should we be able to hypothesize whether the learned firing features should be coarse grained (like devices/tubes) or fine grained (specific sided pleural effusion)?
•	In Figure 3, how is it that SAE-Rad does not include any comparisons with an image prior (the last two sentences in the ground truth report)?
•	Ablation study: are there specific differences in compute/performance with higher expansion factors? Moreover, what is unique about x64 that makes it better than 128 or 32?
•	What was the best performing model that was used? From the ablation it seems like the 64 features? If so, title of the paper should be edited.
•	What is the performance of MAIRA models with the same amount of data as SAERad?
soundness: 3
strengths: •	The paper has a novel implementation of sparse autoencoders for the task of general text generation (radiology reports are the use case in this study).
•	The quality of the writing is high and the paper is quite easy to comprehend.  
•	Multiple examples are provided which helps building a better understanding of the model performance
summary: •This paper introduces SAE-Rad, a methodology that uses SAEs to decompose latent vision transformer representations into features, some of which may be human-interpretable. These features are then used to adapt a pretrained LLM to directly generate radiology reports. Ablation studies are performed to evaluate the sparsity elements of the SAE and some qualitative evaluations of the report quality and counterfactual images is also performed.
weaknesses: •	It is a bit unclear whether the overall goal of the paper is to describe whether SAEs can be used to generate text or to develop a high-quality report generation model. If it is the former, then additional comparisons with more studies image captioning datasets would be better. If it is the latter, then the qualitative reader study would have to be substantially improved, more domain-specific comparisons would be needed, and there could be more domain-specific metrics included. Currently, the paper straddles the area between both fields. 
•	For the claim of modularity of image encoders and LLMs, quantitative comparisons would be needed to solidify this argument. 
•	Metrics choice: Evaluating report quality is challenging, as is well pointed out. Besides RadFact, there are a few new approaches that are being used. The Rexval and Green scores could be particularly useful here.
•	Qualitative investigation shows that randomly selected feature modes depict to human perceivable concepts. This study would be stronger if a more rigorous analysis of these features were to be performed to better understand what the signifiance of these features is.
•	The ablation study uses the radfact score, which has not been extensively substantiated by the community. More common scores here may be better as comparisons since many other models are not benchmarked with this metric.
•	The entire counterfactual section seems very abrupt and well not justified or explained. What is unsupervised segmentation? This section needs substantially more detail or needs to be removed.
•	The reader study seems very superficial, unfortunately. 30 reports are too small to have concluding thoughts from. Were sentences evaluated or the whole results? What constitutes a significant error? Upon reading the appendix, it seems that only 10 reports are analyzed. It is impossible to draw any conclusions from this. Evaluating sentence by sentence also seems worse than evaluation of the whole report.

---

confidence: 5
contribution: 2
initial_rating: 5
presentation: 3
questions: My main concerns are the novelty of the paper (see Q3 and Q4) and the persuasiveness of experimental results (cf. Q5). In addition, it would be better if the authors can improve the readability of their paper, considering the readers unfamiliar with this topic. I am willing to raise my rating if the authors can address my concerns.
soundness: 2
strengths: 1.This work discusses the interpretability of VLMs in radiology report generation tasks that is a hot, interesting, and critical topic.

2.The authors provide an extensive analysis in the paper and appendix.
summary: This paper concentrates on the interpretability of VLMs for radiology report generation. It uses SAE to decompose visual features of an input image and exploits an LLM to distill the entire report into descriptions of SAE features. Finally, by compiling these descriptions, a full report can be obtained.
weaknesses: 1.As far as storytelling is concerned, the motivation for using SAE for model interpretability is not clear to me in the abstract and introduction. I can find some clues in Section 3, but in general it is not obvious, especially to readers unfamiliar with the topic.

2.Overall, the readability of the paper needs further improvement.

3.In terms of methodology, SAE has been well studied in interpreting LLMs (see https://arxiv.org/pdf/2406.04093; https://openreview.net/pdf?id=XkMrWOJhNd). I understand that the focus of this work is on VLMs, and while I like the topic discussed in this paper, I have to say that it does not seem very novel to me. Also, I note that gated SAE from Rajamanoharan et al., 2024.

4.The following paper utilizes the idea of sparse learning (not limited to SAE) to interpret VLMs: https://arxiv.org/abs/2402.10376. The authors do not mention this work.

5.Regarding experiments, in my opinion, the current results are not very convincing, mainly because of the choice of competing methods. I understand that VLM-based methods should be main competitors. But the authors should also compare their model with other non-VLM approaches, because there are already many. I believe that the study of model interpretability should be based on the fact that the model can work well. In addition, there are more VLM-based, open-source methods for radiology report generation, e.g., RaDialog and R2GenGPT, which the authors have not considered. Note that many VLM and non-VLM approaches are open-source, and the authors can reimplement them and compute all the same metrics for them.

---

confidence: 5
contribution: 3
initial_rating: 5
presentation: 2
questions: 1.Why not try replicating these indicators using the open-source methods mentioned above?
2.Why not explore two additional open-source datasets?
3.How can the hallucination issue discussed in the article be resolved?
soundness: 3
strengths: 1.Experimental Section: (1) The qualitative analysis is comprehensive, effectively demonstrating the capabilities of our model. (2) The ablation study is thoroughly designed.

2.The methodology is straightforward and effective, with a clear motivation. This approach directly decomposes image class tokens from a pre-trained radiology image encoder into human-interpretable features, which are then compiled into comprehensive radiology reports.
summary: This paper introduces SAE-Rad, a framework for generating radiology reports using Sparse Autoencoders (SAEs) to decompose image representations into human-interpretable features. SAE-Rad uses a hybrid gated SAE architecture to balance reconstruction accuracy and sparsity, extracting meaningful medical features. The extracted features are transformed into detailed reports by a pre-trained large language model (LLM) without requiring further fine-tuning. Experiments on the MIMIC-CXR dataset show that SAE-Rad achieves competitive performance on radiology-specific metrics, outperforming models like CheXagent with fewer computational resources.
weaknesses: 1.The novelty of the method is not thoroughly demonstrated. While applying Sparse Autoencoders (SAE) in vision-language models (VLMs) represents an innovative effort, the use of SAEs to enhance the interpretability of large language models (LLMs) is already a well-established area. Examples include OpenAI's "Scaling and Evaluating Sparse Autoencoders", which extends SAEs to GPT-4, and Google DeepMind's "Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2".
2.The comparison with existing methods is limited. Table 1 only compares three methods: CheXagent, MAIRA-1, and MAIRA-2, lacking a broader comparison with other open-source methods that generate reports using large language models (LLMs), such as R2GenGPT, RaDialog, and XrayGPT. Although these methods may not utilize the Clinical Metrics evaluation indicators employed in this work, they are open-source, and some provide pre-trained model weights. Why not attempt to replicate these indicators on these open-source methods?
3.The experimental results lack persuasiveness. Although the paper notes that NLG Metrics are not sufficiently comprehensive for evaluating radiological reports, the vast majority of report generation studies still rely on NLG Metrics as the primary basis for evaluation. However, the scores achieved by this work on NLG Metrics are not competitive; with a BLEU-4 score of 1.9, it falls significantly short when compared to other LLM-based radiological report generation efforts, such as RaDialog with a 9.5 and R2GenGPT with a 13.4.
4.The experimental dataset is limited. The quantitative comparisons are conducted using only one public dataset, MIMIC-CXR. However, two other commonly used public datasets for medical report generation are IU-Xray and openi. The paper R2GenGPT, XrayGPT incorporates the MIMIC-CXR and openi datasets.
5.The logical flow of the text is not clear enough. The introduction highlights the primary flaw of existing VLMs as the hallucination issue, yet the paper does not propose an effective method to eliminate hallucinations. Specifically, the introduction of SAE as a solution lacks persuasiveness. Additionally, the introduction states that 'the generated reports a VLM provides may not be faithful to the underlying computations of the image encoder.' Is there factual evidence to support this claim?

Meta-review: The paper introduces SAE-Rad, a novel framework for radiology report generation that combines Sparse Autoencoders (SAEs) with pre-trained language models to produce interpretable, high-quality reports while maintaining computational efficiency. Reviewers acknowledged the potential of applying mechanistic interpretability techniques to vision encoders and appreciated the robust evaluation and interactive tooling provided. However, concerns were raised regarding the limited methodological novelty, as the approach heavily builds on existing SAE research, and the lack of comprehensive comparisons with non-VLM and additional VLM-based baselines. Despite strong empirical results in some metrics, questions persisted about the broader applicability and the system's improvements over existing methods. While the paper presents valuable contributions, the reservations regarding its technical innovation across multiple reviewers unfortunately still remain.

Decision: Reject