# A Review of the Challenges in Deep Learning for Skeletal and Smooth Muscle Ultrasound Images
## 3 Results and Discussion
### Segmentation Architecture in Skeletal Muscle  
The network architecture used for the skeletal muscle segmentation requires specific layer treatments to maximize deep learning performance. It is necessary to combine three different layer types in the network architecture for upsampling with a feature representation phase and score map reconstruction phase in segmentation (Figure 5A). [PERSON] et al. (2019) built a CNN to segment the rectus muscle using a deconvolutional layer, a concatenation layer, and a convolutional layer plus a rectified linear unit (ReLU) activator. The convolution layer was the core building block of the CNN. The layer's parameters consisted of a small number of learnable filters (or kernels) that were wide and deep [44]. The CNN network consisted of two stages: the feature map extraction function and the reconstruction of the score map. The deconvolutional layer multiplies a single input with a learned filter to generate an output patch. In this situation, the patches that overlap in the output are simply added to create the result [45]; however, the deconvolution would cause the loss of the high-resolution information in images [34; 46]. The concatenation layer connects the enlarged score map with the corresponding feature map. It fuses the feature maps in the previous pooling layers or convolution layers with the current feature maps in the deconvolution layer to overcome high-resolution information loss [47; 48]. The skip connection layer design could capture more multiscale contextual information to improve segmentation accuracy [19]. The convolution layer integrated the image information after the concatenation layer. The performance of the proposed model was evaluated and compared to state-of-the-art muscle segmentation methods. During voluntary contraction, a real-time ultrasound image sequence was obtained from the rectus femoris muscle to establish an automated tracing method for the cross-sectional area and shows high matrix performance. A deconvolutional layer without a concatenation layer can be used for segmentation in skeletal muscles [49] (Figure 5B,D); however, when using the deconvolutional layer in upsampling, it needs to go through the fully connected layer between the convolutional layer and the ReLu activator to consider cropping strategies for the preprocessing. [PERSON] et al. used a DCNN in 2018 to look at the shape of a fiber pathway and muscle location with a fully connected layer to combat overfitting using DCNN architecture [33]. Like [PERSON] et al., [PERSON] et al. determined the mechanics and pathological conditions of a muscle-tendon complex in 2020 and monitored the myotendinous junction in ultrasound images using a RAN. The RAN used a deconvolutional layer without a concatenation layer [36]; however, the RAN was not fully connected using the cropping strategy for regions of interest (ROI) in preprocessing. The ultrasound image on the myotendinous junction had characteristic distribution data with noise in different slice hyperplanes which can reduce segmentation accuracy. Therefore, a U-shaped segmentation network in each layer consists of the convolutional layer, followed by a batch normalization layer and a ReLu [50; 51]. In skeletal muscle ultrasound imaging, the use of a RAN presents advantages in applications that use single-shot techniques and transfers learning to solve the inherent muscle ultrasound imaging problems. A RAN was proposed with a unified framework, transfer learning, and region cropping strategies to address the challenges of poor image quality, scarce data, and data imbalance; however, RANs are not currently used in areas other than CNNs, which are often used in the context of skeletal muscles. Figure 5: Segmentation in skeletal muscles; (**A**) CNN architecture with three different layers in the upsampling for rectus femoris segmentation. (**B**) DCNN architecture with fully connected layer for segmenting fiber orientation. (**C**) U-Net architecture with two different layers in upsampling for neck muscle pattern segmentation.