Processing math: 100% Skip to article View PDF Download full issue View Open Manuscript Outline Abstract Keywords Abbreviations 1. Introduction 2. Materials and methods 3. Results 4. Discussion Human rights Informed consent and patient details Disclosure of interest Funding Author contributions Authors credits Acknowledgements References Show full outline Figures (6) Diagnostic and Interventional Imaging Volume 101, Issue 12, December 2020, Pages 789-794 Original article Computer developments Abdominal musculature segmentation and surface prediction from CT using deep learning for sarcopenia assessment Author links open overlay panel P.Blanc-DurandabcdJ.-B.SchirattidK.SchuttedP.JehannodP.HerentdF.PigneureO.LucidarmefgY.BenaceurhA.SadatehA.LucianieO.ErnstiA.RouchaudjM.CrezeklA.DallongevillemN.BanastenM.CadioI.BousaidpN.LassaulqS.Jegoud Show more Add to Mendeley Share Cite https://doi.org/10.1016/j.diii.2020.04.011 Get rights and content Under an Elsevier user license open archive Abstract Purpose The purpose of this study was to build and train a deep convolutional neural networks (CNN) algorithm to segment muscular body mass (MBM) to predict muscular surface from a two-dimensional axial computed tomography (CT) slice through L3 vertebra. Materials and methods An ensemble of 15 deep learning models with a two-dimensional U-net architecture with a 4-level depth and 18 initial filters were trained to segment MBM. The muscular surface values were computed from the predicted masks and corrected with the algorithm's estimated bias. Resulting mask prediction and surface prediction were assessed using Dice similarity coefficient (DSC) and root mean squared error (RMSE) scores respectively using ground truth masks as standards of reference. Results A total of 1025 individual CT slices were used for training and validation and 500 additional axial CT slices were used for testing. The obtained mean DSC and RMSE on the test set were 0.97 and 3.7 cm2 respectively. Conclusion Deep learning methods using convolutional neural networks algorithm enable a robust and automated extraction of CT derived MBM for sarcopenia assessment, which could be implemented in a clinical workflow. Previous article Next article Keywords TomographyX-ray computedDeep learningMuscular body bassSarcopeniaConvolutional neural networks (CNN) Abbreviations 2DTwo-dimensionalBMIBody mass indexCNNConvolutional neural networkCTComputed tomographyDSCDice similarity coefficientMBMMuscular body massMEMean errorMRIMagnetic resonance imagingRMSERoot mean squared errorSDStandard deviationTAMATotal abdominal muscle area 1. Introduction Body composition including assessment of muscular body mass (MBM) is of major importance in oncology and several chronic diseases. In addition, body composition is associated with treatment toxicity that can affect patient survival [1], [2], [3]. More specifically, sarcopenia, which is defined by a loss of muscular mass and function, is frequently observed in up to 50% of patients with cancer [4]. Both physical examination and imaging techniques can be used to diagnose sarcopenia. Anthropometry, which generically refers to body measurements during clinical examination includes body mass index (BMI), skin-fold thickness and body circumference. In addition to physical examination imaging techniques such as computed tomography (CT), dual energy x-ray absorptiometry, ultrasonography and magnetic resonance imaging (MRI) are useful to assess body composition [5]. CT allows using three main indices to diagnose sarcopenia, which are psoas index, total abdominal muscle area (TAMA) and Hounsfield unit (HU) average calculation of body composition through estimation of tissue densities expressed in HU. For TAMA, some thresholds have been suggested [6]. Furthermore, calculation of these indices requires a delineation of muscles (also referred as segmentation) usually at the L3 or L4 vertebra level, which is time consuming and possibly affected by inter-observer variability. To make CT-derived anthropometry clinically applicable, automatic approaches are mandatory. As muscle and fat have different attenuation values on CT, they can be separated from one another using simple thresholding strategies [7]. However, because thresholding may result in a high level of noise, alternative methods have been developed such as thresholding and morphological operations [8], atlas methods [9] or conditional random fields [10]. More recently, convolutional neural networks (CNN), which originated from the deep learning community have been adopted by the radiological community because of their ability to learn spatial features from medical images. They are particularly effective for lesion segmentation or detection tasks in various medical applications [11], [12], [13]. For the specific body composition estimation, CNN have already been employed to automatically identify the axial slice at the mid-L3 level thus facilitating manual segmentation of MBM [14] or impulse the training of a second CNN for segmentation purposes [15]. Several studies have evaluated different CNN architectures on single CT slices for binary segmentations of adipose tissue or muscles [16], [17]. Finally, Weston et al. trained a CNN with a U-net architecture on a large cohort to segment four compartments and used another independent cohort for further validation [18]. The purpose of this study was to build and train a deep CNN algorithm to segment MBM, in order to predict muscular surface from a two-dimensional (2D) axial CT slice at the level of L3 vertebra. 2. Materials and methods 2.1. Study population The CT data were provided as part of the “Sarcopenia Challenge” organized during the 2019 edition of the Journées Françaises de Radiologie, which is the annual meeting of the French Society of Radiology (Société Française de Radiologie). The complete dataset was composed of 1025 axial CT slices for training and validation and was released in two times. A first set was initially available, and a second set was released one month later. Five hundred additional axial CT slices were made available for testing. Inference had to be made in one hour. CT examinations came from multiple French institutions with different acquisition parameters and hardware. All CT examinations and ground truth masks were visually inspected and three of them were excluded because the ground truth masks were not correctly co-registered to their respective original CT examinations. Out of the 1022 axial CT slices that were finally included, 40 were randomly assigned to the validation set with stratification on the quality label. 2.2. Ground truth generation The 2D ground truth masks T, which included abdominal belt and psoas muscles at the level of the mid L3 vertebra were manually annotated by eleven expert radiologists (O.L., N.L, Y.B, A.S., A.L., O.E., L.B., M.C., A.D., N.B. et M.C.), using the public freeware 3DSlicer [19] with a standardized protocol, which included a manual contouring of muscles followed by a fixed thresholding where pixels with attenuation value < −29 HU and > 150 HU were excluded. Each ground truth segmentation was reviewed by a third-party expert (F.P.; with an experience of over 1000 manual segmentations of MBM from CT data for sarcopenia assessment) and was assigned to a quality label ranging from ‘A’ to ‘D’ where: “A” corresponded to perfect segmentation; “B” corresponded to a segmentation with only few pixels in the ground truth mask out of muscles and of which surface was < 0.5 cm2; “C” for masks with a significant number of pixels > 0.5 cm2 were misclassified; “D” when CT were of non-diagnostic quality (noisy images or containing artifacts), if ground truth masks included non-muscular structure or if large muscular portions were missed. 2.3. Model 2.3.1. Pre-processing All available individual CT slices and ground truth masks were resampled to a 512 × 512 tensor of 1 mm2 pixel size, using a linear interpolation of order 3. Pixel values were clipped between −150 HU and 300 HU and normalized using a min-max normalization from [−150 ,300] HU to [0, 1]. Data was stored with 16-bits floating-point precision. 2.3.2. Model architecture We used an ensemble of 15 similar models. Each model was built on a 2D-Unet architecture with a 4-level depth and 18 initial filters. For all resolutions, a block of layers was designed as follows: three convolutional layers with a filter size of 3 and a rectified linear unit (ReLU) activation layer (that breaks the linearity of the model). The number of filters in the encoding path was doubled in each block. Finally, a decoding path mirrored the encoding path for the upsampling part. Each network counted 2,953,409 weights and had a receptive field of 10.7 × 10.7 cm2. Therefore, the 15 models only differed by their weights as they have been trained separately. The average of the 15 outputs (one for each model) was used as final prediction. An overview of the network is shown in Fig. 1. Download : Download high-res image (281KB)Download : Download full-size image Fig. 1. Diagram shows the architecture of the model used. It consists of an ensemble of 15 U-net architectures where the mean of predictions is used as final prediction. 2.3.3. Network training Training was performed on an Ubuntu workstation 16.04 with a 11-Go graphical processing unit GTX1080Ti for 150 epochs of 200 iterations of batch size 12. Initial learning rate was set to 10−3. An adaptive learning rate scheme with a polynomial degree of order 5 was used. Weights were updated with the Adam optimizer. Main augmentation strategies included rotations (−10, +10) and scaling (0.9, 1.1). The Dice similarity coefficient (DSC) was used as a loss function and was back-propagated to the network weights with 1 corresponding to a perfect overlap. 2.4. Post-processing The predicted masks were then resampled back to the image original resolution using a linear interpolation of order 3. Only pixels with a probability of being muscle tissue over 0.5 were kept. The final mask was then computed using a morphological dilation with a squared connectivity of 1, and thresholded at [−29; 150] HU to follow the ground truth procedure. 2.5. Surface estimation From the final mask (P), the predicted surface was estimated as the sum of non-zero pixels multiplied by the pixel size. On the validation set, the mean error (ME) was subtracted from each predicted surface of examinations with quality label A, B or C. This ME was defined as follows: (1) ME=1n∑i=1nSˆi−Si Where Sˆi (resp. Si) denotes the predicted (resp. true) muscle surface for the i-th individual. 3. Results We analyzed the impact of our pre and post-processing on both DSC and root mean squared error (RMSE) in the validation set. As a baseline without preprocessing, DSC and RMSE were respectively 0.93 ± 0.03 (SD) (range: 0.85–0.97) and 4.7 ± 3.1 (SD) (range: 0.0–11.7). DSC and RMSE evolution as a function of the number of U-net model are shown in Fig. 2. They reached a plateau at the tenth model, with further marginal improvement thereafter. It led to a 1.1% DSC rise and a 4.7% RMSE decrease. The post-processing, which included a dilation and a thresholding, improved DSC of 2.3% and decreased RMSE of 17.7%. Finally, after subtracting ME in the validation set to the predicted surface, RMSE decreased from 3.4 cm2 ± 3.1 (SD) (range: 0.1–13.4) to 2.8 cm2 ± 2.7 (SD) (range: 0.0–10.7). Waterfall plots of those different steps leading to the final performances reported are shown in Fig. 3. Download : Download high-res image (210KB)Download : Download full-size image Fig. 2. Graphs show evolution of Dice similarity coefficient (DSC) in the validation set as a function of the number of U-net used in the ensemble of models. A, Mean DSC in the validation set of the 15 trained models with min-max DSC for each epoch. For clarity, we set the epochs axis to 50 instead of 150. The black dashed line corresponds to the 0.95 DSC. B, Evolution of DSC when the number of U-net increased. C, Evolution of root mean squared error (RMSE) when the number of U-net increased. Download : Download high-res image (224KB)Download : Download full-size image Fig. 3. Waterfall plots show impact on Dice similarity coefficient (DSC) (A), and root mean squared error (RMSE) (B), when multiple models are used, with post-processing (including a dilation and a fixed thresholding), and after correction of models bias for RMSE. Some examples of predictions are shown in Fig. 4. DSC in the validation and testing sets were respectively 0.96 ± 0.02 (SD) (range: 0.86–0.98) and 0.97 ± 0.02 (SD) (range: 0.89–0.99). Distribution of DSC for each quality label is displayed in Fig. 5. Download : Download high-res image (387KB)Download : Download full-size image Fig. 4. Left part of figure shows examples of three axial CT slices in the validation set overlaid (in red) with their ground truth masks. Right part of figure shows CT overlaid with in green the true-positive pixels, in blue the false-positive pixels and in red the false-negative pixels. Dice similarity coefficients are 0.971 for A, 0.962 for B, and 0.962 for C. Download : Download high-res image (150KB)Download : Download full-size image Fig. 5. Violin plots show the distributions estimated by kernel density estimate of Dice similarity coefficient (DSC) among the different quality labels of the ground truth masks ranging from better to worse from A to D (x-axis) for the testing set. Medians (white dots), interquartile ranges (thick black line) and 1.5 × interquartile ranges (thin black line) are over-imposed with the kernel density estimate. After post-processing operation, a surface overestimation was observed in the 40 CT slices used for validation (validation set) that is consistent with the dilation that justified our choice to correct our algorithm bias (Fig. 6A). Finally, in the testing set MSE was 13.6 ± 23.6 (SD) (range: 0.0–169.5) and RMSE was 3.7 cm2 ± 2.3 (SD) (range: 0.0–13.0). A scatter plot of predicted surfaces and ground truth surface for the testing set is shown in Fig. 6B. Download : Download high-res image (247KB)Download : Download full-size image Fig. 6. Graphs show scatter plot of predicted area and ground truth area before (blue dots) and after (green stars) algorithm bias correction using linear regression in the validation set A and a scatter plot of predicted area and ground truth area in the testing set B. Of note, on A, most of the blue dots are at the right part of the identity line, indicating surface overestimation. 4. Discussion This study proposed and validated a deep learning algorithm to automatically segment MBM from CT data using an ensemble of U-net architectures. We obtained a DSC of 0.97 ± 0.02 (SD) in the testing set resulting in optimal performances by comparison with prior studies [17], [20]. Using fully connected network Lee et al. reported a DSC of 0.93 for MBM segmentation [17] while using a similar architecture Park et al. reported DSC a 0.96 [20]. More recently, Weston et al. trained and validated a U-net and reached a 0.96 DSC in a large cohort for MBM segmentation but also achieved high performances both for intra-abdominal and subcutaneous fat segmentations [18]. From a methodological point of view, preprocessing was limited as it was only composed of resampling to a 1 mm2 isotropic pixel and a min-max normalization without whitening. We stored our data with 16-bits floating-point precision as Lee et al. demonstrated for the specific body composition segmentation task that neither min-max normalization (referred as windowing) nor a minimum of 256 grey levels (stored on at least 8 bits) impacted DSC in their model [17]. After the post-processing operation, a surface overestimation was observed in the 40 CT slices used as validation set. As a result, we decided to correct the algorithm's bias by subtracting the validation's mean error. We also chose to exclude images with low quality segmentations (denoted by the “D” label quality) for mean error estimation as we believe that ground truth surface may not be reliable enough in that population. We showed that our algorithm was robust to poor label quality. However, the DSC associated with lower quality exams is slightly lower and with higher variance, which can be explained either by an incorrect algorithm prediction or an incorrect ground truth segmentation. Main limitations of this work include the fact that, even if a large dataset for testing was available, a proper independent cohort would be mandatory to validate the algorithm. Also because of the anonymization process nor height nor weight were available which did not allow us to compare performances between groups such as sex or stratified on body mass index. In conclusion, deep learning makes CT derived MBM calculation clinically feasible within the daily radiological workflow. The latter will allow routinely obtaining new biomarker that may provide better diagnosis of sarcopenia and which may benefit to patients within tailored and personalized medicine. Human rights The authors declare that the work described has been carried out in accordance with the Declaration of Helsinki of the World Medical Association revised in 2013 for experiments involving humans. Informed consent and patient details The authors declare that this study does not contain any personal information that could lead to the identification of the patient(s). Disclosure of interest JBS, KS, PJ, PH, SJ, are employees of Owkin. The other authors declare that they have no competing interest. Funding This work did not receive any grant from funding agencies in the public, commercial, or not-for-profit sectors. Author contributions All authors attest that they meet the current International Committee of Medical Journal Editors (ICMJE) criteria for Authorship. All the authors had fully participated to the study and approved the final draft. Authors credits Paul Blanc-Durand: Methodology, Investigation, Software, Validation, Formal analysis, Writing - Original Draft, Writing - Review & Editing Jean-Baptiste Schiratti: Writing - Original Draft, Writing - Review & Editing, Kathryn Schutte: Writing - Original Draft, Writing - Review & Editing, Paul Jehanno: Writing - Original Draft, Writing - Review & Editing, Paul Herent: Writing - Original Draft, Writing - Review & Editing, Frédéric Pigneur: Data Curation, Olivier Lucidarme: Data Curation, Yassine Benaceur: Data Curation, Alexandre Sadate: Data Curation, Alain Luciani: Data Curation, Olivier Ernst: Data Curation, Aymeric Rouchaud: Data Curation, Maud Creuze: Data Curation, Axel Dallongeville: Data Curation, Nathan Banaste: Data Curation, Medhi Cadi: Data Curation, Imad Bousaid: Project administration, Resources, Nathalie Lassau: Project administration, Resources, Conceptualization, Simon Jegou: Methodology, Software, Validation, Formal analysis, Writing - Review & Editing, Supervision. Acknowledgements We would like to thank the Société Française de Radiologie for the opportunity to organize these challenges during the Journées Francophones de Radiologie. We would like to thank Gustave-Roussy for the resources mobilized and the data hosting. We would like to thank the Commission Nationale de l’Informatique et des Libertés (CNIL) for their support. We would like to thank Jean François Raffier and Easys Consulting for the technical support during the challenge. References [1] S.S. Shachar, G.R. Williams, H.B. Muss, T.F. Nishijima Prognostic value of sarcopenia in adults with solid tumours: a meta-analysis and systematic review Eur J Cancer, 57 (2016), pp. 58-67 ArticleDownload PDFView Record in ScopusGoogle Scholar [2] J.J. Hopkins, M.B. Sawyer A review of body composition and pharmacokinetics in oncology Expert Rev Clin Pharmacol, 10 (2017), pp. 947-956 CrossRefView Record in ScopusGoogle Scholar [3] C. Madico, G. Herpe, G. Vesselle, S. Boucebci, D. Tougeron, C. Sylvain, et al. Intra peritoneal abdominal fat area measured from computed tomography is an independent factor of severe acute pancreatitis Diagn Interv Imaging, 100 (2019), pp. 421-426 ArticleDownload PDFView Record in ScopusGoogle Scholar [4] M. Hilmi, A. Jouinot, R. Burns, F. Pigneur, R. Mounier, J. Gondin, et al. Body composition and sarcopenia: the next-generation of personalized oncology and pharmacology? Pharmacol Ther, 196 (2019), pp. 135-159 ArticleDownload PDFView Record in ScopusGoogle Scholar [5] R.D. Boutin, L. Yao, R.J. Canter, L. Lenchik Sarcopenia: current concepts and imaging implications AJR Am J Roentgenol, 205 (2015) W255-266 Google Scholar [6] C.M. Prado, J.R. Lieffers, L.J. McCargar, T. Reiman, M.B. Sawyer, L. Martin, et al. Prevalence and clinical implications of sarcopenic obesity in patients with solid tumours of the respiratory and gastrointestinal tracts: a population-based study Lancet Oncol, 9 (2008), pp. 629-635 ArticleDownload PDFView Record in ScopusGoogle Scholar [7] J. Mendez, A. Keys Density and composition of mammalian muscle Metabolism (1960) https://eurekamag.com/research/024/450/024450136.php (accessed May 17, 2019) Google Scholar [8] S.D. Mensink, J.W. Spliethoff, R. Belder, J.M. Klaase, R. Bezooijen, C.H. Slump Development of automated quantification of visceral and subcutaneous adipose tissue volumes from abdominal CT scans Med Imaging (2011) Computer-aided Diagnosis (Proceedings of SPIE): 79632Q Google Scholar [9] P. Decazes, D. Tonnelet, P. Vera, I. Gardin Anthropometer3D: automatic multi-slice segmentation software for the measurement of anthropometric parameters from CT of PET/CT J Digit Imaging, 32 (2019), pp. 241-250 CrossRefView Record in ScopusGoogle Scholar [10] S. Hussein, A. Green, A. Watane, G. Papadakis, M. Osman, U. Bagci Context driven label fusion for segmentation of subcutaneous and visceral fat in CT volumes (2015) ArXiv151204958 Cs Google Scholar [11] V. Couteaux, S. Si-Mohamed, O. Nempont, T. Lefevre, A. Popoff, G. Pizaine, et al. Automatic knee meniscus tear detection and orientation classification with mask-RCNN Diagn Interv Imaging, 100 (2019), pp. 235-242 ArticleDownload PDFView Record in ScopusGoogle Scholar [12] G. Litjens, T. Kooi, B.E. Bejnordi, A.A.A. Setio, F. Ciompi, M. Ghafoorian, et al. A survey on deep learning in medical image analysis Med Image Anal, 42 (2017), pp. 60-88 ArticleDownload PDFView Record in ScopusGoogle Scholar [13] M. Colevray, V.M. Tatard-Leitman, S. Gouttard, P. Douek, L. Boussel Convolutional neural network evaluation of over-scanning in lung computed tomography Diagn Interv Imaging, 100 (2019), pp. 177-183 ArticleDownload PDFView Record in ScopusGoogle Scholar [14] S. Belharbi, C. Chatelain, R. Hérault, S. Adam, S. Thureau, M. Chastan, et al. Spotting L3 slice in CT scans using deep convolutional network and transfer learning Comput Biol Med, 87 (2017), pp. 95-103 ArticleDownload PDFView Record in ScopusGoogle Scholar [15] C.P. Bridge, M. Rosenthal, B. Wright, G. Kotecha, F. Fintelmann, F. Troschel, et al. Fully-automated analysis of body composition from CT in cancer patients using convolutional neural networks D. Stoyanov, Z. Taylor, D. Sarikaya, J. McLeod, M.A. González Ballester, N.C.F. Codella, et al. (Eds.), 20 Context-Aware Oper. Theaters Comput. Assist. Robot. Endosc. Clin. Image-Based Proced. Skin Image Anal., Springer International Publishing, Cham (2018), pp. 204-213 CrossRefView Record in ScopusGoogle Scholar [16] Y. Wang, Y. Qiu, T. Thai, K. Moore, H. Liu, B. Zheng A two-step convolutional neural network based computer-aided detection scheme for automatically segmenting adipose tissue volume depicting on CT images Comput Methods Programs Biomed, 144 (2017), pp. 97-104 ArticleDownload PDFCrossRefView Record in ScopusGoogle Scholar [17] H. Lee, F.M. Troschel, S. Tajmir, G. Fuchs, J. Mario, F.J. Fintelmann, et al. Pixel-level deep segmentation: artificial intelligence quantifies muscle on computed tomography for body morphometric analysis J Digit Imaging, 30 (2017), pp. 487-498 CrossRefView Record in ScopusGoogle Scholar [18] A.D. Weston, P. Korfiatis, T.L. Kline, K.A. Philbrick, P. Kostandy, T. Sakinis, et al. Automated abdominal segmentation of CT scans for body composition analysis using deep learning Radiology, 290 (2019), pp. 669-679 CrossRefView Record in ScopusGoogle Scholar [19] R. Kikinis, S.D. Pieper, K.G. Vosburgh 3D Slicer: A platform for subject-Specific image analysis, visualization, and clinical support F.A. Jolesz (Ed.), Intraoperative Imaging Image-Guid. Ther., Springer, New York, NY (2014), pp. 277-289 CrossRefView Record in ScopusGoogle Scholar [20] H.J. Park, Y. Shin, J. Park, H. Kim, I.S. Lee, D.W. Seo, et al. Development and validation of a deep learning system for segmentation of abdominal muscle and fat on computed tomography Korean J Radiol, 21 (2020), pp. 88-100 CrossRefView Record in ScopusGoogle Scholar © 2020 Société française de radiologie. Published by Elsevier Masson SAS. Recommended articles Quantification of body composition in renal cell carcinoma patients: Comparing computed tomography and magnetic resonance imaging measurements European Journal of Radiology, Volume 132, 2020, Article 109307 Download PDF View details A two-step convolutional neural network based computer-aided detection scheme for automatically segmenting adipose tissue volume depicting on CT images Computer Methods and Programs in Biomedicine, Volume 144, 2017, pp. 97-104 Download PDF View details Muscle wasting and survival following pre-operative chemoradiotherapy for locally advanced rectal carcinoma Clinical Nutrition, Volume 37, Issue 5, 2018, pp. 1728-1735 Download PDF View details 12 Next Citing articles (13) About ScienceDirect Remote access Shopping cart Advertise Contact and support Terms and conditions Privacy policy We use cookies to help provide and enhance our service and tailor content and ads. By continuing you agree to the use of cookies. Copyright © 2021 Elsevier B.V. or its licensors or contributors. ScienceDirect ® is a registered trademark of Elsevier B.V. ScienceDirect ® is a registered trademark of Elsevier B.V.