# DeepSarc-US: A Deep Learning Framework for Assessing Sarcopenia Using Ultrasound Images
###### Abstract
proposed the masked auto-encoder (MAE), a self-supervised learning approach that includes image inpainting [30]. The MAE paradigm is a self-supervised technique for ViT-based models that enables the network to learn useful information by predicting masked targets. MAE has shown potential for faster training and better generalization of the ViT-based models. In ViT models, feature maps are created based on a single low-resolution image by adopting a fixed-scale windowing step. [PERSON] et al. [32; 33] proposed Swin Transformer V2 (SwinT) that builds hierarchical feature maps by adopting a non-overlapped shifted windowing step into the encoder of vanilla ViT, and it is capable of training with images of up to 1536 \(\times\) 1536 pixels (vanilla ViT refers to ViT model proposed by [29]). SwinT has shown capabilities for learning functional dependencies between features. ViT-based models have demonstrated superior performance over simple CNN-based models in various computer vision tasks as well as US image analysis [34; 35; 36; 37; 38; 39; 40; 41; 42]. However, ConvNext, employing a convolutional architecture, has surpassed ViT-based models in certain contexts of natural image analysis [25]. Notably, neither ViT nor ConvNext have been specifically applied to the measurement of muscle quality as of the current knowledge update. Furthermore, despite the promising performance of ConvNext and ViT-based models compared to simple CNN-based models, the training step is challenging, particularly when there are a lack of sufficient data. In the clinical scope, due to security and privacy policies, the publicly available US data are very limited. Labeled data are especially scarce, since manually annotating medical data is expensive. Therefore, utilizing complex models in clinical settings is quite challenging. To this end, a strategy is proposed for training the complex models on a small set of US images. We aimed to explore the performance of three recent CNN-based and three ViT-based DL models to estimate quadriceps muscle thickness (QMT) based on a limited number of US images using two main strategies that provide a fair comparison of ViT- and CNN-based models. To better comprehend the decisions made by the models, examples of the visualization maps produced by the ViT- and CNN-based models are further examined. These visualization maps can also offer supplementary information that can be used as a guide for practitioners at the time of data acquisition. The contributions of this work are summarized as follows:  
* Three CNN- and three ViT-based models are proposed to estimate QMT using ultrasound images acquired in a clinical setting. * A strategy is proposed for optimizing the training of DL models to estimate QMT more accurately, especially when limited data are available. * The activation maps are explored to provide clinicians with real-time feedback. This feedback can potentially be used to help clinicians collect better US images to help DL models estimate QMT more accurately. * To the best of our knowledge, it is shown for the first time that DL can be used to automatically estimate QMT from US images taken from phased array probes. This paper is organized as follows. In Section 2, first, the data collection and experimental setup are outlined. Then, the results of the proposed QMT estimation strategies and the derived visualization maps are presented in Section 3. Finally, the outcomes of our experiments are concluded in Section 4.