Beyond Structured Attributes: Image-Based Predictive Trends for Chest X-Ray Classification

Authors: katharina  hoebel
jesseba  fernando
william  lotter
Venue: MIDL_io_2024_Conference
Published: 2024-02-01 00:51:01

Abstract: A commonly emphasized challenge in medical AI is the drop in performance when testing on data from institutions other than those used for training. However, even if models trained on distinct datasets perform similarly well overall, they may still exhibit other systematic differences. Here, we study these potential dataset-centric prediction variations using two popular chest x-ray datasets, CheXpert (CXP) and MIMIC-CXR (MMC). While CXP-trained models generally perform better on CXP than MMC test data and vice versa, this performance decrease is not uniform across individual images. We find that image-level variations in predictions are not random but can be inferred well above chance, even for pathologies where the overall performance gap is small, suggesting that there are systematic tendencies of models trained on different datasets. Furthermore, these "predictive tendencies" are not solely explained by image statistics or attributes like radiographic position or patient sex, but rather are pathology-specific and related to higher-order image characteristics. Our findings stress the complexity of AI robustness and generalization, highlighting the need for a nuanced approach that especially considers the diversity of pathology presentation.

Main paper: 
 introduction: 
 Deep learning (DL) models for medical imaging tasks have been shown to be prone to performance degradation on data from unfamiliar sources, such as from previously unseen institutions (Yu et al., 2022; Zech et al., 2018) . Among the factors contributing to the reduction in performance are covariate and concept shifts (Cohen et al., 2020; Zhang et al., 2023) ; the former occurs due to shifts in the distribution of features between the training and testing datasets, while the latter occurs when the relationship between the input features and target variables changes. A substantial body of research has investigated the relationship between categorical attributes such as label distribution and patient demographics with performance and fairness gaps in the context of generalization (Pooch et al., 2020; Seyyed-Kalantari et al., 2020; Larrazabal et al., 2020; Ahluwalia et al., 2023; Seyyed-Kalantari et al., 2021; Glocker et al., 2023) . Notably, shifts in these structured attributes across datasets often only account for a fraction of the generalization gaps, leaving room for understanding the scope of features that contribute to a lack of generalization (Wu et al., 2021) . Traditionally, these generalization analyses are also often "model-centric", focusing primarily on assessing how the performance of one model varies across different datasets. While this approach offers insights into model robustness and adaptability, it may not fully capture the nuanced effects that the training domain has on the model behavior.Here, we employ a "dataset-centric" approach to study how models trained on different domains behave when evaluated on the same dataset, helping to isolate the influence of training data. We specifically use two public chest x-ray datasets, CheXpert (Irvin et al., 2019) and MIMIC-CXR (Johnson et al., 2019) , given their popularity and the high prevalence of commercial products for this modality. We first demonstrate that dataset-centric performance gaps exist for all pathologies in these datasets. Nonetheless, we find high variation in predictions at the image level independent of these gaps. We demonstrate that this variability is not simply noise, but can be predicted significantly above chance, where these "predictive tendencies" are specific to each pathology, cannot be explained by standard structured attributes alone, and seem to be encoded in high-level image characteristics. 
datasets: 
 We used two public chest x-ray datasets: 1) CheXpert (CXP) (Irvin et al., 2019) with 224,314 images from 65,240 patients from Stanford Hospital patients and 2) MIMIC-CXR (MMC) (Johnson et al., 2019) consisting of 377,110 images from 65,379 patients at Beth Israel Deaconess Medical Center. Both datasets include metadata and labels for 14 radiological findings. When performing subgroup analysis, we considered structured attributes of radiographic projection (view), patient race, sex, and age. We included race subgroups of Asian, Black, and White patients in alignment with Gichoya et al. (2022) and 'AP', 'PA', and 'Lateral' views.We trained deep learning models for two distinct tasks as described below. These models are termed Pathology Prediction Models (PPMs) and Comparative Dataset Models (CDMs). Accordingly, we split each dataset into four subsets, consisting of 50% for training the PPMs, 20% for training the CDMs, 10% for validation for both model types, and 20% for testing for both model types. These partitions were performed on a patient-level to ensure that all radiographs from one patient were included in the same partition.Pathology Prediction Models (PPMs) The PPM models are trained according to the standard task of predicting the presence of pathologies. We train these models using the following 12 pathology labels: 'Atelectasis', 'Cardiomegaly', 'Consolidation', 'Edema', 'Enlarged Cardiomediastinum', 'Fracture', Lung Lesion', 'Lung Opacity', 'Pleural Effusion', 'Pleural Other', 'Pneumonia', 'Pneumothorax'. For robustness, we mapped uncertain labels (-1) as missing. We trained models separately on CXP and MMC and refer to them as CXP-PPM and MMC-PPM.Comparative Dataset Models (CDMs) The CDMs are designed to discern whether, for a given pathology, an x-ray is more likely to yield a relatively higher prediction from the CXP-PPM or the MMC-PPM. We refer to this difference in the PPM predictions as the predictive tendency. To generate the CDM training labels for a given dataset (Figure 1 ), we first rank the PPM predictions for the CXP-and MMC-PPMs for a given pathology p, with the normalized rankings denoted as Φ p . This step helps to mitigate calibration discrepancies between the two PPMs. Subsequently, we compute the predictive tendency s p (x i ) for image x i and pathology p as follows:s p (x i ) = Φ p CXP (x i ) -Φ p M M C (x i ) (1)The resulting values are binarized into discrete labels, denoted as s ′ p (x i ), using the median value from the CDM-training dataset as the binarization threshold, which results in a balanced distribution of labels. Each CDM was trained on a pathology-specific subset of the full dataset. For example, for pneumothorax, we compute the predictive tendency labels for all images with pneumothorax and use these labels to train a pneumothorax-specific CDM. The trained CDM outputs an inferred predictive tendency ŝp ′ (x i ) to estimate whether a model trained on CXP or MMC outputs a relatively higher pathology prediction for that image. Model Architecture and Training For both training tasks (PPM and CDM), we employed DenseNet121 models (Huang et al., 2016) , pre-trained on ImageNet. Models were trained for 50 epochs and training was performed separately on each dataset. Final weights were selected based on the highest performance, as determined by AUROC, on the validation dataset. For each model configuration (PPM/CDM and CXP/MMC), we train three identical models using different random seeds to increase the robustness of the analysis. Unless specified otherwise, reported performance metrics represent the average across the three models. When using the PPMs to create training labels for the CDMs, the PPM predictions are averaged across the three random seeds and subsequently ranked. Models are implemented and trained using the TorchXrayVision library (Cohen et al., 2021) . Overall, our goal was to develop models representing standard architectures, hyperparameters, and image preprocessing, where more information can be found in the Appendix A.To study the importance of image characteristics for a CDM's ability to learn predictive tendencies, we applied the following image transformations: 1) Pixel permutations: to disrupt the spatial information without affecting intensity distributions by randomizing the position of each pixel within an image; 2) Frequency filtering: to selectively remove low and high-frequency content (see Appendix A for more details).To test the association between the distribution of predictive tendencies and categorical structured attributes (view, race, sex), we first performed a Kruskal-Wallis test (Kruskal and Wallis, 1952), followed by a post-hoc Dunn test (Dunn, 1964) . We use ϵ 2 (King and Minium, 1981) to determine effect sizes. Associations of the predictive tendency with patient age are tested using the Spearman correlation coefficient. We apply Bonferroni correction to correct for multiple comparisons. Statistical analysis was conducted using python 3.9 with SciPy 1.10 and scikit-posthocs 0.8.1 (Virtanen et al., 2020) . 
pathology prediction model performance and performance gaps: 
 The PPMs trained on CXP and MMC achieved overall AUROC scores of 0.89 and 0.93 respectively (micro-average across all 12 pathologies), with AUROCs for each pathology detailed in Appendix B, Table 2 . The performances of the PPMs are comparable to those reported in the existing literature, despite using only 50% of the available data for training (Seyyed-Kalantari et al., 2020; Pham et al., 2019) .The generalization gap for a model f and performance metric R is traditionally defined by the difference: R(f A , D A )-R(f A , D B ), where D A and D B are distinct, mutually exclusive domains, and f A denotes the model developed on D A . We focus on a dataset-centric approach for comparing model performance: R(f A , D A ) -R(f B , D A ), where the models are trained separately on D A and D B and evaluated on the same domain. This dataset-centric perspective shifts the focus to the training datasets to offer a more direct comparison of models on the same data points.The performance gaps between in-domain and out-of-domain models across all 12 pathologies are depicted in Figure 2A . In-domain models consistently outperformed out-of-domain ones, though the extent of these gaps varied by pathology. Fracture and pneumothorax exhibited the largest gaps, while effusion and atelectasis had the smallest. Notably, pathologies with higher PPM model performance tended to show narrower gaps (Figure 2B ).Beyond aggregate performance differences, we studied the model behavior at the individual image level. In this analysis, we focused on three pathologies due to their varying performance gaps: pneumothorax, pneumonia, and pleural effusion. We first visualized the image-level prediction scores between the different models. As illustrated in Figure 2C and D, notable image-level variations were observed, even for pleural effusion which exhibits high predictive performance and a relatively smaller performance gap. To quantify these differences, we introduce the concept of predictive tendency (s) (Section 2.2) that measures the difference in the ranked prediction scores between PPMs trained on different datasets. In the analysis that follows, we analyze these predictive tendencies for images that are labeled positive for each of the three selected pathologies. Structured Attributes To examine whether the observed predictive tendencies are random or reflect systematic features, we first explored whether the tendencies are correlated with structured attributes: patient demographics (race, sex, age) and x-ray radiographic view. Akin to prior analysis of generalization gaps with these datasets, we find that these attributes can only explain a small fraction of the observed predictive tendencies. Across the categorical attributes (race, sex, view), the attribute generally showing the highest association is view (Figure 3 and Appendix D, Figure 9 ). Nonetheless, this effect was really only visually apparent for pleural effusion with an ϵ 2 effect size of 0.18 in MMC (with 0 being chance and 1 corresponding to all variance explained). The next highest effect size was 0.06 for patient sex in the CXP dataset for pneumothorax, with generally low visually apparent differences in the distribution of predictive tendencies (Appendix D, Figure 9 ). All other ϵ 2 effect sizes across patient sex and race were below 0.01. Correlations with patient age were also generally low, with an Spearman correlation coefficients ranging from 0.02-0.21 across datasets and pathologies (Appendix D, Table 3 and Fig. 10 ). Intensity Distributions As the structured attributes only explain a small fraction of the observed variance, we next investigated a possible correlation between the predictive tendencies and first-order image statistics, namely the distribution of pixel intensities. If there were such a correlation, one would expect differences in the intensity histograms between images that have a predictive tendency towards MMC versus towards CXP. While we do observe differences in the histograms between the CXP and MMC datasets themselves, the variation between images with different predictive tendencies was negligible compared to these disparities between the datasets, as shown in Appendix E. Notably, the differences in intensity histograms between datasets were consistently larger than the differences between predictive tendencies even after adjusting for structured attributes.As neither structured attributes nor first-order image statistics could explain the predictive tendencies, we next asked whether they are determined by patterns that DL models could recognize. To this end, we trained a separate set of deep learning models termed Comparative Dataset Models (CDMs) (Figure 1 and Section 2.2). CDMs were trained separately for each dataset and for each of the three selected pathologies to avoid introducing the dataset source as a potential shortcut and to enable comparisons across pathologies. Surprisingly, we find that the predictive tendencies can indeed be predicted well above chance levels, with AUROCs between 0.74 and 0.85 (pneumonia and pleural effusion MMC-CDMs, see Table 1 ). Notably, CDMs trained on CXP or MMC showed meaningful crossdataset performance, suggesting generalizable predictive patterns (grey values in Table 1 ). With this highly non-trivial CDM performance, we next explored what types of information these models have learned. We first examined exemplary images, comparing those that received high predictions from the CDMs against those with low predictions. This was followed by examining the associations between CDM predictions and structured attributes. These analyses aligned with our previous findings regarding the predictive tendencies, where radiographic view had a moderate association with CDM predictions but other associations were much less apparent. More details and visual representations can be found in Appendices F and G.Next, we assessed whether the models generalize across pathologies, i.e., if a CDM model trained on images with pneumothorax could also predict the predictive tendencies for images with pleural effusion. Interestingly, we find that this is not the case. As shown in Appendix H, Tables 5 and 6 , the models exhibit nearly random AUROC performance with a range of 0.43-0.60. These results further suggest that the predictive tendencies are pathology-specific and cannot be explained by low level statistical differences.Subsequently, we tested how other image characteristics might influence CDMs' ability to discern predictive tendencies by applying targeted image transformations, retraining the CDMs on these transformed images and assessing the impact on CDM performance. To examine the role of spatial relationships, we randomized pixel positions in the x-rays, disrupting the spatial structure but preserving the intensity distribution. This pixel permutation significantly reduced CDM performance, dropping AUROCs to near-chance levels, between 0.55 and 0.64 (Appendix H, Table 7 ), again aligning with our previous findings that predictive tendencies are not linked to low-level image statistics.We then assessed the importance of image frequencies by filtering out high or low frequencies, following the method of Gichoya et al. (2022) who used this explainability approach in the context of models trained to predict patient race. When applied to the CDMs, the filtering variably affected performance but consistently showed that CDMs rely on both frequency ranges (Figure 4 ). Notably, CDMs maintained non-trivial performance even when features are largely not perceptible to the human eye, qualitatively similar to the results of Gichoya et al. (2022) for predicting patient race and suggesting that both sets of models may rely on a mix of statistical features for their respective tasks.  
discussion: 
 Despite a wealth of research into DL model performance gaps, particularly in chest x-ray analysis, a significant portion of these gaps remain unexplained (Wu et al., 2021) . In this study, we adopt a "dataset-centric" perspective to examine performance gaps between DL models trained on different datasets, revealing that these gaps vary across pathologies and are influenced by their representation in the datasets. Beyond performance however, a key contribution of our work is introducing the concept of "predictive tendency" -the notion that models trained on different datasets can exhibit systematic differences in predictions regardless of overall performance. Our analysis suggests that these tendencies in two popular datasets are pathology-specific and depend on higher-order image statistics that are not fully captured by structured attributes. These results especially emphasize the heterogeneity of disease presentation (Oakden-Rayner et al., 2019) and label creation, where variations in representation across datasets and even radiologist/clinic-specific interpretation practice could potentially contribute to our findings. Beyond showing that the tendencies can be predicted using separate deep learning models, we envision avenues where this approach can enable context-dependent ensembling strategies of multiple models, and tailored model selection to individual patient and image profiles more generally. Altogether, our analysis highlights the importance of a nuanced view of generalization, emphasizing the need to address context-specific biases while leveraging this knowledge increase performance robustness across all patients.  
 
 Appendix:  


Decision: The paper studies robustness of models trained on different datasets for the same task and shows systematic differences which cannot be revealed by overall performance, but can be predicted based on pathology and image characteristics. The reviewers agreed on the clarity of the paper and the importance of the investigation. The reviewers had some questions, which appear to have been all addressed in the rebuttal, and the reviewers are in agreement on the quality of the paper. I’m happy to recommend acceptance.

Reviews: review: paper_summary: paper_summary: The authors develop a method to analyse classifiers’ discrepancy in term of performances when trained and tested on two different chest X-ray datasets. They introduce the « predictive tendency » metric that compare the ranked outputs of two classifier trained on different data but tested on the same population. This metric allows the author to train a second model that try to predict this metric from an image. They finally analyse the characteristics of images with high and predictable discrepancy.
main_review: main_review: I think that this kind of analyse are very valuable for the community, and this paper does it right. I think that it deserve an Oral presentation given the extensive analysis and the clear presentation
strength_weakness: strength_weakness: No major weakness

The paper might increase a little bit the litterature review. For example this paper [3] do a dataset centric analysis by splitting by sex.

Other paper talked about bias in chest x-ray [2]
[1] Ben Glocker et al., Risk of Bias in Chest Radiography Deep Learning Foundation Models
[3] Agostina J Larrazabal et al, Gender imbalance in medical imaging datasets produces biased classifiers for computer-aided diagnosis
questions: questions: na
paper_summary: paper_summary: This work adopts a “dataset-centric” perspective to examine performance gaps between DL models trained on different datasets, revealing that these gaps vary across pathologies and are influenced by their representation in the datasets. The key contribution of the work is introducing the concept of “predictive tendency”, which indicates that models trained on different datasets can exhibit systematic differences in predictions regardless of overall performance.
main_review: main_review: I think this paper gives a unique view of the AI model across different datasets. The predictive tendency is a good concept to evaluate the prediction difference between models trained on different datasets. Based on this idea and the response, I think this paper deserves accepted.
strength_weakness: strength_weakness: 1. There is a lack of ablation experiments to the extent to which these influencing factors affect the predictions of the model, such as controlling the effects of age, gender, etc., and observing the prediction results under different disease conditions.
2. Predictive tendency can find the prediction difference between two datasets. However, which factors contribute to this difference are not better quantified. For example, given a CXR image and two prediction results from different models, How much does the age factor contribute to the tendency?
questions: questions: 1. Provide details about CDMs to explain the predictive tendency.
2. Provide some specific examples to explain your results.
paper_summary: paper_summary: This study sheds light on systematic differences observed in models trained on different datasets. The investigation focuses on chest X-ray classification using two popular datasets: CheXpert and MIMIC-CXR. The findings reveal that variations at the image level are not arbitrary but are correlated with specific pathologies, indicating consistent tendencies in models trained on distinct datasets. These observed "predictive tendencies" cannot be solely attributed to image statistics or factors such as radiographic position or patient gender. Instead, they are pathology-specific and linked to higher-level image attributes. The study underscores the importance of diversifying pathology representation to enhance the robustness and generalizability of AI models.
main_review: main_review: This paper addresses an important aspect for deploying robust and fair AI model for chest X-ray classification. They investigate attributes beyond structure and demographics to better understand the models. They introduce the concept of "predictive trends" and emphasize the importance of diversifying pathology representation to enhance the robustness and generalizability of AI models.
strength_weakness: strength_weakness: - The distinction between your study and Gichoya et al. 2022 is not sufficiently elaborated.
Gichoya Judy Wawira, et al. "AI recognition of patient race in medical imaging: a modelling study." The Lancet Digital Health 4.6 (2022): e406-e414.
questions: questions: 1. Highlight differences between your work and Gichoya, Judy Wawira, et al. "AI recognition of patient race in medical imaging: a modelling study." The Lancet Digital Health 4.6 (2022): e406-e414.

Comments: title: Invitation to reply to authors
comment: I thanks the authors for their answers. I will take them into account.
Thank you for your response and clarifications, I am satisfied with the replies and have no further comments.
Thank you for your response and clarifications, I am satisfied with the replies and have no further comments.
Dear reviewers,

The authors have prepared responses to your comments, which you should now be able to see in OpenReview. We encourage you to reply to their comments, and where necessary, adjust your rating. Please do so before the 27th of March.
__Order of Figures__ We thank the reviewer for this practical advice and have moved Figure 3 to the methods section in cross-reference with Section 2.2.

__Incomplete sentence__ We apologize for this oversight and have added the missing word. The complete sentence reads: *“If there were such a correlation, one would expect differences in the intensity histograms between images that have a predictive tendency towards MMC versus those with a predictive tendency towards CXP.”* 

__Code availability__ We thank the reviewer for this suggestion. To enhance reproducibility and transparency, we will make our code available on GitHub upon acceptance and added the link to the manuscript: https://github.com/lotterlab/xray_generalization.git 

__Differences between our work and Gichoya et al.__ We thank the reviewer for the opportunity to clarify the differences between the work by Gichoya et al. and ours. Gichoya et al. focus on the ability of deep learning (DL) models to predict race from medical images; our work focuses on image-based factors that contribute to performance differences of DL classifiers trained on different datasets. Thus, while both works study AI models in the context of medical image characteristics, they have different goals and focuses. Importantly, we find that our identified predictive tendencies do not correlate with structured attributes such as patient race, age, and sex (Section 3.2 and Appendix D). Beyond these structured attributes, we performed several additional analyses and explainability techniques to understand the image characteristics underlying the predictive tendencies. Among others, we performed an experiment inspired by Gichoya et al. using low and high-pass filters to modify the frequency content of the images. This approach, however, diverges in its application since the AI models are developed for different tasks. We have now clarified these differences in the last paragraph in the Results section.
__1. Questions regarding the Comparative Dataset Models (CDM)__ Thank you for the opportunity to clarify aspects of our Comparative Dataset Models (CDMs) and their outputs. The CDMs are trained to predict the predictive tendency, i.e., the relative difference in pathology-specific prediction scores between models trained on different datasets. Thus, the output of the CDM is an estimate of the predictive tendency calculated based on the pathology predictions of the Pathology Prediction Models (PPMs). 
We modified the figure describing the CDMs to more explicitly define their outputs and how they relate to the predictive tendencies. We have also moved this figure to the Methods section to accompany the written description in Section 2.2. We have additionally expanded upon this written description to accompany the mathematical formulation. 

__2. Provide examples to explain the results__ Thank you for this suggestion. We have now included example images that received the highest and lowest CDM prediction values for each dataset in Appendix F. The example images align with our other results in that there is large heterogeneity amongst these images. We identified several contexts where particular radiographic views are overrepresented, which are consistent with correlations between radiographic views and the predictive tendencies we describe in Section 3.2.

__3. Factors affecting predictions, such as age and gender__ We appreciate the reviewer's insight on how structured attributes may contribute to our findings. In the original submitted version of the manuscript, we had indeed quantitatively assessed how age, gender, race, and radiographic view may contribute to the predictive tendency. We have now expanded this analysis to explicitly assess if these factors are associated with the CDM predictions. We describe both of these aspects below. 

__Predictive tendency__ We quantitatively assessed the association between the predictive tendency and structured attributes using two approaches. First, we assess the statistical significance of the relationships using a Kruskal-Wallis test. Second, we use epsilon squared as a standardized metric to quantify the magnitude of an attribute’s impact on the predictive tendency. These results are described in Section 3.2, Figure 2, and Appendix D (Figures 9, 10, and Table 4). Importantly, we find very low associations between these structured attributes and the predictive tendencies, which we have now expanded upon in Section 3.2. 

__Comparative Dataset Models__ Additionally, we have performed a new analysis on the impact of the same set of structured attributes on the CDM predictions (inferred predictive tendencies). These findings largely mirror the effects described in the paper for the predictive tendencies, where these structured attributes can only explain a small fraction of the CDM predictions. Overall, we found that the radiographic view had the strongest associations with the CDM predictions, though the effect sizes (epsilon squared) remained low, with a range between 0.07 and 0.32 across the pathologies. Patient sex had a statistically significant association with the CDM predictions for pneumonia and pleural effusion but with very small effect sizes (epsilon squared of 0.006 and 0.02, respectively). Patient race and age did not have a statistically significant association with the CDM predictions. We have included this new analysis in Appendix G. 

While we are not aware of other works studying our defined predictive tendencies, our findings are consistent with prior work studying the effects of structured attributes on pathology prediction performance. Studies by Seyyed-Kalantari et al. [1] and Wu et al. [2] revealed that only a minor portion of chest X-ray classifiers’ performance discrepancies between the CheXpert and MIMIC datasets stem from structured attributes such as age, gender, and race.

__4. Preprocessing of images__ As our goal is to study generalization in the context of standard AI approaches, we adopted best practices for chest X-ray image preprocessing as performed in prior studies. Aligning with methodologies used by works such as Cohen et al. [3] and Pooch et al. [4], we adopt similar image sizes and normalize the grayscale intensity values to a range of -1024 to 1024 to ensure a standardized model input. We have clarified this motivation in Appendix A.

[1] Seyyed-Kalantar et al. (2020). CheXclusion: Fairness gaps in deep chest X-ray classifiers. Biocomputing 2021
[2] Wu et al. (2021). Explaining medical AI performance disparities across sites with confounder Shapley value analysis. Proceedings of Machine Learning Research. 
[3] Cohen et al. (2020). On the limits of cross-domain generalization in automated X-ray prediction. Proceedings of Machine Learning Research
[4] Pooch et al. (2020). Can we trust deep learning based diagnosis? The impact of domain shift in chest radiograph classification. International Workshop on Thoracic Image Analysis
We thank the reviewer for their positive review and helpful comments, especially for the observation that the work of Larrazabal et al. represents a data-centric analysis of performance gaps. We have now extended the literature review to include all of the mentioned references as well as additional work in this area, as suggested by the reviewer.
We sincerely thank all the reviewers for their time and fair and balanced evaluations. We address each reviewer's feedback in a separate comment below the respective review. To ensure clarity and avoid redundancy, we have structured our responses to address comments by subject area and summarize the feedback (bolded) before offering detailed replies. Additionally, citations from the manuscript are highlighted in italics. We look forward to a constructive and engaging discussion during the discussion period.