Interpretable Uncertainty-Aware Deep Regression with Cohort Saliency Analysis for Three-Slice CT Imaging Studies

Authors: nouman  ahmad
johan  öfverstedt
håkan  ahlström
joel  kullberg
Venue: MIDL_io_2024_Conference
Published: 2024-01-30 14:31:24

Abstract: Obesity is associated with an increased risk of morbidity and mortality. Achieving a healthy body composition, which involves maintaining a balance between fat and muscle mass, is important for metabolic health and preventing chronic diseases. Computed tomography (CT) imaging offers detailed insights into the body’s internal structure, aiding in understanding body composition and its related factors. In this feasibility study, we utilized CT image data from 2,724 subjects from the large metabolic health cohort studies SCAPIS and IGT. We train and evaluate an uncertainty-aware deep regression based ResNet-50 network, which outputs its prediction as mean and variance, for quantification of cross-sectional areas of liver, visceral adipose tissue (VAT), and thigh muscle. This was done using collages of three single-slice CT images from the liver, abdomen, and thigh regions. The model demonstrated promising results with the evaluation metrics – including R-squared ($R^2$) and mean absolute error (MAE) for predictions. Additionally, for interpretability, the model was evaluated with saliency analysis based on Grad-CAM (Gradient-weighted Class Activation Mapping) at stages 2, 3, and 4 of the network. Deformable image registration to a template subject further enabled cohort saliency analysis that provide group-wise visualization of image regions of importance for associations to biomarkers of interest. We found that the networks focus on relevant regions for each target, according to prior knowledge. The source code is available at: \url{https://github.com/noumannahmad/dr_3slice_ct}.

Main paper: 
 introduction: 
 Understanding metabolic health is important for understanding the risks and mechanisms of type 2 diabetes (T2D) and cardiovascular disease (CVD) (Stefan and Schulze, 2023) . Body composition is associated with metabolic health. The analysis of body composition involves examining the quantities and distribution of both fatty and non-fatty tissues, including adipose tissue, muscle, liver, and bone. Precise measurement of body composition is crucial for understanding the mechanisms and progression of cardiometabolic diseases, particularly T2D and CVD, thereby aiding in their prediction and prevention (Sneed and Morrison, 2021) . Non-alcoholic fatty liver disease (NAFLD) (Gaggini et al., 2013; Ariya et al., 2021) , fat tissues and muscles, especially visceral adipose tissue (VAT), the fat stored around abdominal organs are associated with metabolic risk (Janochova et al., 2019; Vasamsetti et al., 2023) . NAFLD and VAT can cause insulin resistance, a key factor in the development of T2D and metabolic disturbances, increasing the risk of T2D, coronary artery disease, and CVD (Samaras et al., 2010) . Computed tomography (CT) is an imaging technique that allows detailed analysis of body composition (Kullberg et al., 2017; Ahmad et al., 2023) . SCAPIS and IGT, are two large Swedish cohort studies that include CT imaging for detailed studies of body composition and its link to cardiometabolic disease and Chronic Obstructive Pulmonary Disease (Bergström et al., 2015) . The IGT study focuses on investigating the influence of gut microbiota on glucose dysregulation and cardiometabolic risk (Molnar et al., 2023) .In this feasibility study, we propose to use the ResNet-50 network, as illustrated in Figure 1 for the quantification of variables such as liver, VAT, and thigh muscle areas from a collage of three single-slice CT images, as applied to the SCAPIS and IGT cohort studies. SCAPIS and IGT include three single-slice CT imaging data for body composition analysis. The acquisition of only a few slices minimizes ionizing radiation (mean 0.2 mSv) (Bergström et al., 2015) compared to volumetric CT scans while still providing body composition information. For uncertainty awareness, the network predicts both mean and variance, providing an estimate of its ability to predict the target value. For interpretability, Grad-CAM is used at different stages of the network to generate saliency maps highlighting important regions for each prediction, and deformable image registration (on the CT images) is employed to align the resulting saliency maps into a common template space, enabling a cohort saliency map to identify shared important regions across the entire cohort. The network exhibits promising results in predicting the selected targets according to multiple evaluation measures, suggesting the viability of deep regression to predict clinical parameters from images acquired using the three single-slice CT imaging protocol. The saliency maps highlight areas expected to be of high importance to the prediction targets, reinforcing trust in both the cohort saliency analysis methodology and the proposed deep regression method. 
background and related work: 
 Deep learning techniques (Elhakim et al., 2023) are increasingly favored due to their proficiency in automatic feature extraction and decision-making. These advanced methods are effective in segmenting and measuring the liver, different types of fat, thigh muscles, and other tissues and organs. Beyond segmentation, these techniques can be used for classification and aid in the estimation and prediction of body composition parameters, age, BMI, and other non-imaging parameters and biomarkers, as described in studies including MR images from the UK Biobank Study (Langner et al., 2020 (Langner et al., , 2022;; Starck et al., 2023) . From CT images of the thorax and abdomen, a ResNet-18 network has previously been used for age prediction (Kerber et al., 2023) .Numerous deep learning network architectures have been developed for a variety of tasks, with ResNet-50 (He et al., 2016) being widely utilized due to its skip connections facilitating efficient and effective training. To address the challenge of limited data in medical imaging, transfer learning methods are a powerful family of techniques for leveraging large amounts of training data in one domain to solve tasks in a different domain where data is limited. Since deep learning models typically require large amounts of data for effective training, using transfer learning (Kim et al., 2022) to retrain models with medical images can substantially enhance performance, proving particularly beneficial in this context.The integration of deep learning in medical imaging decision-making poses challenges related to trustworthiness due to the black-box nature of deep learning systems, but these can be partially mitigated by enhancing the interpretability and uncertainty awareness of these systems. For network interpretability, Grad-CAM (Gradient-weighted Class Activation Mapping) (Selvaraju et al., 2016) is one of the commonly used methods in deep learning that highlights features and spatial regions of importance for a network's prediction by computing the gradient of each feature w.r.t. a selected target class/neuron.Image registration is a technique in medical imaging that aligns (multiple) images to a common reference space (Fox et al., 2008) . This standardization enables comparative studies, such as Imiomics analysis (Strand et al., 2017) and the generation of cohort saliency maps (Langner et al., 2020) . In (Ekström et al., 2020) , a graph-cut-based method for image registration was proposed. Building on this methodology, in (Jönsson et al., 2022) , a registration technique for PET-CT images was proposed, incorporating anatomical and tissue-wise masks to enhance the robustness and accuracy of the registration. 
dataset overview and ethical statement: 
 This study used data from two large cohort studies, SCAPIS and IGT. The primary objective of the SCAPIS study is to explore T2D, CVD, and COPD (Bergström et al., 2015) . SCAPIS included 30,154 participants, comprising both males and females aged between 50 and 64 years who volunteered to participate. The SCAPIS study data was collected in six university hospitals across Sweden in the time period from 2013 to 2018. For this study, a randomly selected subset of data from participants recruited in Gothenburg was used.The IGT (Molnar et al., 2023) study is a parallel cohort to SCAPIS, with a focus on individuals at high risk of developing T2D. This study particularly investigates the role of the gut microbiota in the dysregulation of glucose and the consequent development of CVD. The IGT cohort comprised 1,965 participants with varying forms of glucose dysregulation. Both SCAPIS and IGT cohorts were examined using a common CT imaging protocol.Our study received approval from the Swedish Ethical Review Authority (Dnr 2021-05856-01), and all participants provided their informed consent in written form.All CT images were acquired according to a predefined protocol, featuring an image dimension of 512×512×1, a field of view of 500mm, and a slice thickness of 5mm. For the analysis, three specific CT slices were utilized, which we refer to as: liver, abdomen, and thigh. The liver slices were reconstructed from lung scans. The abdomen slice was positioned above the crista edge and centrally aligned with the L4 vertebra. The thigh slices were captured at a midpoint between the outer edge of the acetabulum and the joint surface of the knee.In this study, 2,724 subjects were analyzed. The SCAPIS subjects included 500 male participants (age 58.3 ± 4.3 years, BMI 27.9 ± 4.2 kg/m 2 ) and 452 female participants (age 58.5 ± 4.3 years, BMI 27.2 ± 5.4 kg/m 2 ). The IGT cohort comprised 784 males (age 58.4 ± 4.5 years, BMI 28.2 ± 3.9 kg/m 2 ) and 988 females (age 57.8 ± 4.5 years, BMI 27.4 ± 4.6 kg/m 2 ). Age and BMI values are presented as mean ± standard deviation (std).Image segmentation was used for two main purposes in this work. Firstly, segmentation masks corresponding to the liver, VAT, and thigh muscles were generated by CNN-based segmentation models (UNET++) (Ahmad et al., 2023) . These masks were used to estimate the area (cm 2 ) of the liver, VAT, and thigh muscle, parameters which were used as prediction targets for this feasibility study. The UNET++ models demonstrated very good performance, with high Dice scores for the masks: liver (0.994), VAT (0.937), and thigh muscle (0.996), indicating their suitability as sources of reliable values for the experiments liver, VAT, and thigh muscle in this work. Secondly, the liver and thigh muscle masks, as well as spleen, SAT, and skeletal muscle masks (having Dice scores of 0.993, 0.990, and 0.988 respectively) were used to facilitate deformable image registration.In addition to the organ and tissue-specific masks, we also created body masks for the liver, abdomen, and thigh slices to remove CT tables appearing in the FOV by applying a threshold to CT intensities above -190 HU. These body masks were used to remove elements lacking relevance from the images, aiding both registration and deep regression.Finally, to facilitate the image registration, several thresholding-based masks were generated. We extracted vertebra masks in the liver and abdomen slices, where intensities above 200 HU were thresholded, followed by a morphological opening operation to remove noise. In the abdomen slice, the skeletal muscle mask was utilized to generate an intrasubcutaneous adipose tissue (ISAT) mask. For the thigh slices, masks for cortical bone and lean tissues were generated by thresholding above 200 HU for bone and between -29 HU and 150 HU for lean tissues (Broder, 2011; Mitsiopoulos et al., 1998) .For deep regression, the ResNet-50 network, as illustrated in Figure 1 (a), was trained for quantification of liver area, VAT area, and thigh muscle area (separately per target). Initially trained on the ImageNet database with 1000 classes, we adopted a transfer learning approach (Kim et al., 2022) , by modifying the last top layer to customize the network for our specific tasks. To make the network uncertainty-aware, rather than predicting a single target value (point prediction), the network was configured to produce two output values: the mean and variance (uncertainty estimation) of a Gaussian distribution. Initially, the dimensions of each CT image were 512×512×1. Deep regression was applied to all three images as a collage. This was done for two reasons: (i) to input all image information to the network, and (ii) to test if the model could learn to use the relevant slice (according to prior knowledge) for each prediction target. To form these collages, three slices of liver, abdomen, and thigh were pre-processed to remove the tables and stacked together as illustrated in Figure 1 , resulting in a collage with dimensions of 1004×1004×1. The stacked images were resized and converted into a three-channel input of 512×512×3, to make the images compatible in format with the pre-trained network.The network was trained using 10-fold cross-validation, with slight data augmentation consisting of random shifts (up to 16 pixels both horizontally and vertically). We also performed additional experiments with other augmentation techniques including rotation, shear, re-scaling, and elastic deformation. However, the performance dropped when these augmentation techniques were included in preliminary experiments, so we omitted them from the main experiments of this work.We used a batch size of 16, a specialized mean-variance training loss function (Lakshminarayanan et al., 2016) , as described in more detail in the appendix A, and the Adam optimizer (Kingma and Ba, 2014) with a learning rate of 0.0001. The network was trained for 10,000 iterations, and the learning rate was reduced to 0.00001 after 8,000 iterations. The regression pipeline was implemented using the PyTorch framework (Paszke et al., 2019) .As a pre-study, we assessed several networks including VGG-16 (Karen Simonyan, 2014), DenseNet121 (Huang et al., 2017) , EfficientNet (Mingxing and Le., 2019) , a Transformerbased network (Vaswani et al., 2017) , a graph neural network (Naman and David, 2019) , and also ResNet-18/50. ResNet-50 was chosen based on its overall best performance across the three targets.To assess common regions of interest across the entire cohort studies, a cohort saliency map analysis was performed. We applied Grad-CAM to generate a saliency map for each prediction. We then transformed each Grad-CAM saliency map into a common reference space using a deformation field obtained by image registration. These saliency maps were then aggregated by taking the voxel-wise mean saliency, and we refer to these aggregate saliency maps as cohort saliency maps. They were visually evaluated to observe the common regions highlighted by the network across the entire cohort. We performed this analysis by computing the Grad-CAM saliency maps at stages 2, 3, and 4 of the network.The registration pipeline (Ahmad et al., 2024) used to facilitate the cohort saliency analysis involved a one-step process utilizing multiple channel inputs simultaneously, including raw input CT images and their corresponding masks. During the registration, a higher weight was assigned to the CT images compared to the masks. To configure the registration method, we select 8 pyramid levels and regularization weight maps used were 0.05 for air, adipose, and soft tissues, while the bone was assigned a weight of 0.1. Image weights were defined as 0.5 for liver and thigh slices and 0.4 for abdomen slices. For body masks, weights were set to 0.1, 0.15, and 0.2. The registration process used the sum of squared differences as an objective function and a displacement field transformation model (Jönsson et al., 2022) . For optimization, a fast graph-cut-based method was used (Ekström et al., 2020) .The reference spaces were chosen to be as representative as possible for each group of slices to be registered. To achieve this an approach using z-scores of explicitly quantified organ and tissue depots were used and different template images were selected for the different slices, for males and females in IGT and SCAPIS resulting in a total of 12 different templates (3 slices × 2 sexes × 2 studies). For the liver slice, the sum of the z-scores of liver and spleen areas was minimized. For abdomen slices, the sum of the z-scores of VAT, subcutaneous adipose tissues (SAT), and skeletal muscle areas was minimized. Similarly, for thigh slices, the sum of the z-scores of thigh SAT and muscle areas was minimized.The network was trained for quantification of the liver area(cm 2 ), VAT area(cm 2 ), and thigh muscle area(cm 2 ) by evaluating target predictions in terms of mean and variance (uncertainty) scores. The efficacy was assessed by measuring the quality of fit using the coefficient of determination (R 2 ), which delineates the proportion of variance accounted for by the network. The mean absolute error (MAE), defines the disparity between actual and predicted values. The experimental results were performed on Intel (R) Xeon(R) W-2133 CPU at 3.60 GHz with 32 CPU RAM and Nvidia RTX Ti with 11GB of GPU RAM. 
results and discussion: 
 The proposed method demonstrated good results overall, as shown in Table 1 , with high R 2 and low MAE across targets. Notably, VAT and thigh muscle areas showed high R 2 scores and low error rates. The uncertainty estimation, represented as variances, generally indicated confidence in the predictions, except for a high variance in the liver area predictions for the IGT cohort, possibly due to anatomical variation across subjects. For comparison, the model were also trained with mean squared error (MSE) as the loss function and the results are listed in appendix B (Table 2 ), where we observed lower performance compared to the uncertainty-aware approach.A comparison of IGT predictions and actual measurement scatter and Bland-Altman plots are illustrated in appendix C (Figure 3 ) and for SCAPIS in appendix E (Figure 5 ). In the scatter plot, the x-axis is labeled actual score (cm 2 ), and the y-axis is labeled predicted score (cm 2 ). The green crosses represent individual data points, and the red line is the line of best fit. The predictions fall close to the regression line, suggesting a strong linear relationship between the actual and predicted scores. In the Bland-Altman plot the green crosses represent individual data points, showing the difference between the predicted and actual scores against their average.The Grad-CAM cohort saliency map analysis, illustrated in Figure 2 for IGT, for SCAPIS illustrated in appendix D (Figure 4 ). Additionally, a cohort saliency map was generated using random template selection for the IGT study to analyze the robustness of the cohort saliency map to the selected template. The results were found to be consistent with the chosen z-score based template, as demonstrated in the appendix F (Figure 6 ), highlighting the same targeted regions such as liver, VAT, and thigh muscle areas in both male and female cohorts, as for the z-score based template. Stage 2 includes the initial convolutional layers, batch normalization, ReLU activations, max pooling, and the first two stages of the ResNet-50. Early stages generally capture low-level features such as edges, colors, and basic textures. The highlighted regions include the liver and thigh muscle areas simultaneously, suggesting the presence of shared feature information at that particular stage. We believe that this might indicate a statistical relationship between these areas, used by the network in the prediction of the liver and thigh muscle areas. To evaluate this we performed Pearson correlation analysis on explicit measurements of liver and thigh muscle areas, with correlations of 0.24/0.21 for males/females in SCAPIS, and 0.30/0.38 for males/females in IGT confirming the plausibility of the relationships observed in the saliency analysis. For the VAT target, Grad-CAM highlights the VAT areas. In Stage 3, the network started to capture more complex and high-level features. These layers can recognize more abstract patterns, which are likely more relevant for identifying target regions.Stage 4 includes all stages up to but excludes the last two layers (which are average pooling and the final fully connected layer). The features at this stage of the network are even more high-level and abstract than stage 3. These layers capture complex patterns with high relevance for solving the prediction task. The higher representational power at this stage could result in more robust and stronger highlighting of relevant regions, at the cost of further reduced spatial resolution. We notice that the spine in liver slices is light up, which the model is picking up on additional areas beyond the target. It could be due to their complex patterns or liver size varying across subjects that the model has learned to recognize as important. Further, this could be a case of the model being too sensitive to certain features, leading to a broader area of highlight than intended.A strength of this study was that we could demonstrate the feasibility of the cohort saliency by highlighting target regions of interest in the three slice collages. A limitation of the choice of target measurements is that they were all derived from single-slice measurements, requiring no interaction between the information in the different slices to successfully solve the task. To extend this to more complex multi-slice target measurements is an interesting future research avenue. Another limitation is that, although we present cohort saliency results, deep regression is less explainable compared to models based on explicit image-based measurements. Furthermore, successful deep regression requires extensive data; segmentation approaches may be advantageous for explicit feature measurement. 
conclusion: 
 In this feasibility study, an uncertainty-aware deep regression pipeline with associated Grad-CAM cohort saliency mapping has been successfully developed and evaluated for body composition measurement predictions from three single-slice CT collages. 
 
 Appendix:  


Decision: This paper proposes a framework to predict the area of different image regions (e.g. of the liver) from CT slices using deep regression. The methodology is a fairly straightforward application of existing approaches, but the work is clinically well motivated and implemented carefully. 

Overall, the paper received very mixed reviews. Unfortunately, the most critical reviewer did not respond to the rebuttal, but I had a careful look at the paper, review, and rebuttal myself. I believe the authors addressed the raised concerns adequately. Two reviewers were concerned about the limited methodological novelty. As this is more of a translational paper, I would tend to disregard this, although I agree the technical contributions are limited. 

While not groundbreaking, I believe this is a sound application paper with an interesting and clinically relevant application. As in my opinion, the authors responded well to the negative reviews, I tend to follow the positive reviewers and can recommend acceptance of the paper.

Reviews: review: paper_summary: paper_summary: A 2D slice-wise CT based analysis of liver, abdomen and thigh regions to estimate the adiposity of the patients to provide quantification of metabolic health of patients is performed using a relatively large dataset 2,724 patients. A 2D Resnet classifier was created to predict the mean and standard deviation of the metabolic volume to provide a measure of uncertainty in the estimate of the adiposity when using 3 specific slices corresponding to the iver, abdomen, and the thighs. GraCam approach was combined with a cohort saliency analysis to show the which regions were implicated in making the prediction. 

The methodology is incremental. Key details of the analysis including how the resnet was trained to predict mean and standard deviation to model uncertainty, how were the individual 3 slices selected, registration method, selection of the canonical reference etc are all missing. There may be some conceptual novelty regarding the automated estimation of the metabolic health.
main_review: main_review: * The methodology is incremental.
* Key details of the analysis including how the resnet was trained to predict mean and standard deviation to model uncertainty, how were the individual 3 slices selected, registration method, selection of the canonical reference etc are all missing. 
* How robust is the method to the selection of the individual slices? 
* How robust is it to the selection of the canonical patient reference. 
* How was the registration performed between cohorts if discrete sets of slices from each patient was used in the analysis.
strength_weakness: strength_weakness: * The methodology is incremental.
* Key details of the analysis including how the resnet was trained to predict mean and standard deviation to model uncertainty, how were the individual 3 slices selected, registration method, selection of the canonical reference etc are all missing. 
* How robust is the method to the selection of the individual slices? 
* How robust is it to the selection of the canonical patient reference. 
* How was the registration performed between cohorts if discrete sets of slices from each patient was used in the analysis.
questions: questions: * The methodology is incremental. What is the novelty here? 
* Key details of the analysis including how the resnet was trained to predict mean and standard deviation to model uncertainty, how were the individual 3 slices selected, registration method, selection of the canonical reference etc are all missing. 
* How robust is the method to the selection of the individual slices? 
* How robust is it to the selection of the canonical patient reference. 
* How was the registration performed between cohorts if discrete sets of slices from each patient was used in the analysis.
paper_summary: paper_summary: The authors present a study where the body composition (measuring thigh muscle, liver, and VAT areas) is inferred using three CT slices at pre-specified locations jointly fed to a ResNet50 network. The model includes an additional output to the regression head, giving an uncertainty estimate assuming a Gaussian error distribution. The study also includes a population-based saliency map analysis for which Grad-CAM outputs were registered to a common space.
Cross-validation was used to evaluate the method on two datasets (SCAPIS and IGT).
main_review: main_review: Thank you for the replies to my comments and the other reviewer's suggestions, and the effort you put into revising the manuscript and adding additional experiments. I believe it has improved and important points have been made clearer.
strength_weakness: strength_weakness: - I would appreciate it if you could compare your approach and results to the previous works you mention and cite in the section on related works. For someone not too familiar with this specific task, it would be great to be able to gauge the performance you report
- The uncertainty estimates are only discussed very briefly. It would be great if you could add some information if, e.g., the uncertainties match the deviations you got measured by the MAE.
- In my opinion, the title is a bit over-promising regarding the interpretability. Judging from the title, I was expecting a non-standard interpretable approach apart from using Grad-CAM.
questions: questions: - Can you comment on the clinical impact of the accuracy/error you are making with your approach? I.e., how does this compare to what is useful in practice and what others have achieved?
- Did you consider looking into Bland-Altman plots to see if there is a constant offset in your regression results?
paper_summary: paper_summary: This is a feasibility study that uses CT imaging to understand body composition and its related factors. The study utilizes deep learning techniques to segment and measure different types of tissues and organs, and to aid in the estimation and prediction of body composition parameters. The study also uses saliency maps to highlight areas of high importance to the prediction targets, which reinforces trust in the methodology and the proposed deep regression method. The study provides group-wise visualization of image regions of importance for associations to biomarkers of interest. The study is important for understanding metabolic health and preventing chronic diseases.
main_review: main_review: This is a feasibility study that uses CT imaging to understand body composition and its related factors. The study utilizes deep learning techniques to segment and measure different types of tissues and organs, and to aid in the estimation and prediction of body composition parameters. The study also uses saliency maps to highlight areas of high importance to the prediction targets, which reinforces trust in the methodology and the proposed deep regression method. The study provides group-wise visualization of image regions of importance for associations to biomarkers of interest. The study is important for understanding metabolic health and preventing chronic diseases.

The authors gave answers to my comments in weakness quite properly, especially in the comparison of other methods even if it is only descirption instead of a quantitative one. considering the potential outcome in the future, I will raise my evaluation for this paper from the inital one ( 4: Weak accept) by one step higher.
strength_weakness: strength_weakness: One of the main weaknesses of the paper is that it lacks a comparison with existing state-of-the-art methods. While the paper does compare its proposed method with a few baseline methods, it does not compare its results with other recently published methods in the field. This limits the ability to assess the true novelty and potential impact of the proposed method.

Another weakness of the paper is that it does not provide a detailed explanation of the limitations of the proposed method. Why did the authors use Deep Regression with ResNet-50 instead of another models--the inclusion of such defense will improve this paper. 

Furthermore, the paper could benefit from a more detailed explanation of the methodology used for cohort saliency analysis. For example, how the saliency maps were generated and how they were used to identify regions of interest will reinforce this paper.

Also, Conclusion section of the paper seems insufficient.
questions: questions: The lack in comparison study might be due to their originality and novelty. However, the authors might consider a similar study and their methods and results for the comparison purpose.
paper_summary: paper_summary: The paper train and evaluate an uncertainty-aware deep regression based on ResNet-50 for quantification of cross sectional areas of liver, visceral adipose tissue (VAT), and thigh muscle, with outputs its prediction mean and variance. The model was evaluated using R2 and MAE as metrics for predictions, showing high R2 and low MAE across targets. Additionally, the model was evaluated with saliency analysis based on Grad-CAM. Deformable image registration to a template subject was used to get group-wise cohort saliency analysis.
main_review: main_review: I want to express my appreciation for addressing my comments as well as the suggestions from the other reviewer. The revised manuscript  incorporated additional experiments which enhanced the clarity of important points. But still I feel the paper lacks originality and novelty.
strength_weakness: strength_weakness: The paper doesn't show much novelty. The paper seems just apply ResNet-50 and Grad-CAM saliency map to another application, to   quantify the liver area, abdomen VAT area. In addition, the paper fail to do comprehensive experiments such as why trainning one for all rather than 3 networks etc.
questions: questions: Could you explain why you trained one network for you? I would like to the authors to perform the comparison of once-for-all network with 3 networks respectively for liver area, VAT area and thigh muscle area quantification. The comparison includes the R2, MSE, and saliency maps. In addition, train another network only to output mean, and do comparison. Last, try some other data augmentation and see whether it benefit the network performance.

Comments: title: raise my evaluation from the initial one
Gentle reminder of rebuttal deadline
comment: The authors gave answers to my comments in weakness quite properly, especially in the comparison of other methods even if it is only descirption instead of a quantitative one. considering the potential outcome in the future, I will raise my evaluation for this paper from the inital one ( 4: Weak accept) by one step higher.
Question: Could you explain why you trained one network for you? I would like to the authors to perform the comparison of once-for-all network with 3 networks, respectively, for liver area, VAT area, and thigh muscle area quantification.

Reply: Thank you for raising this point. The training of a model for a single individual target is likely to enhance the model performance, by avoiding interference between the features learned to solve each individual task. Though sometimes multi-task learning can be beneficial, the targets of interest here are derived from different slices, and therefore unlikely to benefit from shared features. Conversely, training the model with multiple targets tends to increase network complexity and may reduce performance. Also training each model individually allows for easier evaluation and interpretation of results. We can assess each model's performance independently, providing clearer insights into how well each target is being addressed, and what features contribute to the prediction.

Furthermore, as we mentioned in the manuscript, this is a proof-of-concept study that allows us to highlight relevant target regions in the image with precision while interpreting the output, as we know what to expect/where to look. All in all, training a multi-target network for this purpose could introduce noise for each individual target prediction, and potentially decrease the performance.


Question: In addition, train another network only to output mean, and do comparison.
	
Reply: Thank you for your feedback. We have now incorporated results from models trained with MSE into the appendix, see Appendix B.  The results from the proposed model utilizing the mean-variance training loss function show slightly improved MAE error and R2 outcomes compared to those obtained with MSE, thus providing both uncertainty estimates that we can use to assess how robust our predictions are, as well as obtain higher prediction performance.

Question: Last, try some other data augmentation and see whether it benefit the network performance.

Reply: For data augmentation, we have experimented with various techniques including rotation, shear, scaling, and elastic deformation. However, the network's performance did not improve from inclusion of these data augmentations, but the performance decreased. A statement about this observation has been added to the manuscript (see section 3.3.1).
Question: The paper could benefit from a detailed explanation of the usage of the proposed method.

Reply: The proposed method is specifically tailored for SCAPIS and IGT, employing three single-slice images to establish correlations between imaging and non-imaging data. By training a regression model, it predicts the liver area, fat, and muscle composition of a subject, utilizing these predicted measurements for further analysis in the cohort study with follow-up assessments. 

Our pipeline is a proof of concept for the liver area, VAT, and thigh muscle. The cohort saliency analysis based on Grad-CAM has shown that our model accurately identifies these important regions during cohort saliency analyses, paving the way for integrating imaging data with other biomarkers.

In the future, we plan to use this approach to study, in detail, deep-learning based links and associations between imaging data and disease-related biomarkers of interest, for example, related to diabetes, age, and cardiovascular disease. 

Question: The paper could benefit from a more detailed explanation of the limitations of the proposed method.

Reply: While we do present cohort saliency results, it's important to note that our approach may be less explainable compared to models relying on explicit image-based measurements. Additionally, the reliance on deep regression requires extensive data. Although segmentation approaches offer advantages for explicit feature measurement, it's worth acknowledging that our current methodology serves as a proof of concept, as mentioned in the paper. In future work, we aim to incorporate additional parameters such as BMI and age, which may not be feasible with segmentation techniques. 

Question: The paper could also include a more thorough comparison with existing state-of-the-art methods.

Reply: In our paper, we highlighted that it is a proof of concept tailored for SCAPIS and IGT studies that use three-slice CT images. Our goal was to create a pipeline for these specific cohorts. Early on, we assessed several networks, including VGG-16, EfficientNet, DenseNet121, transformer based model, graph neural networks, and ResNet-18. ResNet-50 was chosen as our baseline for its optimal performance for all targets. In future work, we aim to enhance the pipeline by incorporating more non-imaging parameters and biomarkers, as ResNet-50 shows the best overall performance for various targets. A statement about this observation has been added to the manuscript (in Section 3.3.1).
Question: Can you comment on the clinical impact of the accuracy/error you are making with your approach? I.e., how does this compare to what is useful in practice and what others have achieved?

Reply: Thank you for your question regarding the clinical impact of our model's accuracy. It is important to note that our study is primarily contributing to epidemiological research rather than clinical application. We have conducted a proof of concept study of a method for body composition analysis using three-slice CT images.

Our model's accuracy in terms of metrics such as R-squared and mean absolute error has shown promising results using 3 slices of imaging data for epidemiological research, as it can provide a solid foundation for understanding body composition changes over time for SCAPIS and IGT. While the direct clinical impact is not the goal, the precision of our model is important for accurate longitudinal follow-up measurements.

The relatively good performance suggests that we will be able to use the developed technology in further research related to predictions related to research on type 2 diabetes, cardiovascular disease, and associated risk factors. At this high level of accuracy/precision, the variance induced by imaging (single slice only and variance in positioning) would also contribute a significant proportion of the total method variance.

Question: What others have achieved?
Reply: Our study demonstrates the quantification of visceral adipose tissue (VAT) and thigh muscle area using 3 single-slice CT images. Specifically, we achieved an R² of 0.997 for VAT and 0.998 for thigh muscle area in the SCAPIS cohort, with mean absolute errors (MAE) of 3.431 cm² for VAT and 1.593 cm² for thigh muscle area. In contrast, the author in paper [1], in their supplementary material, reported R² values of 0.994 for VAT and 0.993 for thigh muscle, with MAEs of 0.122 L and 0.162 L, respectively, using deep regression with ResNet50 on UK Biobank data.
It's important to note the different units of measurement (cm² in our study vs. litres in the referenced study) and the imaging modality used (CT vs. MRI). Despite these differences, both studies underscore the high accuracy and reliability of deep learning models in body composition analysis. Our results further build upon these findings by demonstrating the utility of CT imaging limited to 3 single slices only and highlighting the model's precision, as evidenced by lower MAEs in predicting VAT and thigh muscle areas, showing the effectiveness of our methodology in body composition.

Reference:
[1]. Langner, Taro, et al. "Uncertainty-aware body composition analysis with deep regression ensembles on UK Biobank MRI." Computerized Medical Imaging and Graphics 93 (2021): 101994. DOI. https://doi.org/10.1016/j.compmedimag.2021.101994

Question: Did you consider looking into Bland-Altman plots to see if there is a constant offset in your regression results? 

Reply: Thank you for your valuable suggestion to incorporate Bland-Altman plots to assess the presence of a constant offset in our regression results. Following your advice, we have now included Bland-Altman plots in Appendix C and Appendix E, given the page limit constraint for the main manuscript.
Question: The methodology is incremental. What is the novelty here?

Reply: Thank you for your query regarding the novelty of our methodology. Our approach is distinctive in that it integrates an uncertainty-aware deep regression model using ResNet-50 with cohort saliency mapping via Grad-CAM, specifically for body composition analysis with three-slice CT images—a method not previously explored in metabolic health studies with multiple single slices. 

There is also novelty in the image registration and in the evaluation of the cohort saliency maps at different stages of the ResNet-50 network. Details on critical components of the image registration have now been added to this paper. (The complete details of the image registration and its evaluation are in a paper under review.)


Question: Key details of the analysis including how the resnet was trained to predict mean and standard deviation to model uncertainty, how were the individual 3 slices selected, registration method, selection of the canonical reference etc are all missing.

Reply: We appreciate the reviewer's request for further methodological details. Our uncertainty-aware deep regression model was trained using a specialized loss function that allows the network to predict both the mean (predicted score) and variance (uncertainty in prediction) for each target parameter. This loss function, adapted from Lakshminarayanan et al. (2016), is particularly effective for regression models that require the estimation of parameters under the assumption of Gaussian-distributed errors. For more details, please refer to Appendix A, where we describe the loss function.

Regarding the three CT slices, we realize that this might have been a source of misunderstanding. The three CT slices collected are determined during the imaging sessions according to the SCAPIS and IGT study imaging protocols. We now have clarified this more in Figure 1 and its caption as well as in the introduction the first time the three slices are mentioned.  

The registration method is now described in more detail. For a more in-depth explanation of the selection of the reference images please see sections 3.2 and 3.3.


Question: How robust is the method to the selection of the individual slices?

Reply: As previously clarified, the methodology described in this paper does not involve specific methods for selecting individual slices. Instead, the study is based on a dataset consisting of three predetermined slices from the SCAPIS and IGT cohorts, with standardized positions in the liver, abdomen, and thighs. As we do not have access to volumetric data in these studies we can unfortunately not analyze the precision and robustness to change in slice positioning.

 Question: How robust is it to the selection of the canonical patient reference.

Reply: We thank the reviewer for this very good suggestion. We have now redone the cohort saliency analysis three times per sex with randomly selected template slices. The results from these new experiments highlighted similar regions for all three targets as our proposed z-score-based approach. The results are added in Appendix Section F.

Question: How was the registration performed between cohorts if discrete sets of slices from each patient was used in the analysis.

Reply: Inter subject registrations are always challenging. The standardized positioning of the imaged slices was indeed critical for the standardization of the image content. The registrations in this work are always done within cohorts and within the same sex.
Dear authors,

This is just a gentle reminder that the rebuttal deadline is coming up. I encourage you to take this opportunity so you can engage with the reviewers in the upcoming discussion phase.

Best wishes,
AC