# Ultrasound segmentation analysis via distinct and completed anatomical borders

Vanessa Gonzalez Duque, Alexandra Marquardt, Yordanka Velikova, Lilian Lacourpaille, Antoine Nordez, Marion Crouzier, Hong Joo Lee, Diana Mateus, Nassir Navab

► To cite this version:

Vanessa Gonzalez Duque, Alexandra Marquardt, Yordanka Velikova, Lilian Lacourpaille, Antoine Nordez, et al.. Ultrasound segmentation analysis via distinct and completed anatomical borders. International Journal of Computer Assisted Radiology and Surgery, 2024, 19 (7), pp.1419-1427. 10.1007/s11548-024-03170-7 . hal-04772928

HAL Id: hal-04772928

https://nantes-universite.hal.science/hal-04772928v1

Submitted on 26 May 2025

**HAL** is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
ORIGINAL ARTICLE

<figure><img src="image_2.png" /></figure>

Ultrasound segmentation analysis via distinct and completed
anatomical borders

Vanessa Gonzalez Duque<fnref n="1" />,<fnref n="2" />,<fnref n="3" />,<fnref n="4" /> Ⓓ · Alexandra Marquardt<fnref n="1" />,<fnref n="2" /> · Yordanka Velikova<fnref n="1" />,<fnref n="2" /> · Lilian Lacourpaille<fnref n="4" /> · Antoine Nordez<fnref n="4" /> · Marion Crouzier<fnref n="4" /> · Hong Joo Lee<fnref n="1" /> · Diana Mateus<fnref n="3" /> · Nassir Navab<fnref n="1" />,<fnref n="2" />

Received: 7 March 2024 / Accepted: 30 April 2024
© The Author(s) 2024

**Abstract**

**Purpose** Segmenting ultrasound images is important for precise area and/or volume calculations, ensuring reliable diagnosis and effective treatment evaluation for diseases. Recently, many segmentation methods have been proposed and shown impressive performance. However, currently, there is no deeper understanding of how networks segment target regions or how they define the boundaries. In this paper, we present a new approach that analyzes ultrasound segmentation networks in terms of learned borders because border delimitation is challenging in ultrasound.

**Methods** We propose a way to split the boundaries for ultrasound images into distinct and completed. By exploiting the Grad-CAM of the split borders, we analyze the areas each network pays attention to. Further, we calculate the ratio of correct predictions for distinct and completed borders. We conducted experiments on an in-house leg ultrasound dataset (LEG-3D-US) as well as on two additional public datasets of thyroid, nerves, and one private for prostate.

**Results** Quantitatively, the networks exhibit around 10% improvement in handling completed borders compared to distinct borders. Similar to doctors, the network struggles to define the borders in less visible areas. Additionally, the Seg-Grad-CAM analysis underscores how completion uses distinct borders and landmarks, while distinct focuses mainly on the shiny structures. We also observe variations depending on the attention mechanism of each architecture.

**Conclusion** In this work, we highlight the importance of studying ultrasound borders differently than other modalities such as MRI or CT. We split the borders into distinct and completed, similar to clinicians, and show the quality of the network-learned information for these two types of borders. Additionally, we open-source a 3D leg ultrasound dataset to the community https://github.com/Al3xand1a/segmentation-border-analysis.

**Keywords** Ultrasound segmentation · Anatomical borders · 3D US dataset

**Introduction**

Ultrasound (US) imaging is essential in medical diagnostics,
providing non-invasive visualization of internal organs. Its

advantages include versatility, safety, and portability, mak-
ing it a first choice in fields like obstetrics, cardiology, and
oncology [1]. Segmenting specific regions in ultrasound aids
in diagnosing medical conditions by analyzing organs like
the liver, heart, or kidneys. However, segmenting ultrasound
images can be challenging due to factors like noise, speck-
les, shadows, and low contrast [2]. Unlike CT or MRI, which
show consistent tissue distributions, ultrasound depends on
the absorption and reflection of acoustic waves at tissue inter-
faces. As a result, delineation of anatomical boundaries can
be particularly challenging in regions exhibiting low signal
intensity or in the presence of artifacts.

Many medical practitioners tackle US segmentation chal-
lenges by first classifying, then focusing on a region of
interest, and finally separating structures based on tissue
interfaces [3]. Specifically, during ultrasound segmentation,

Vanessa Gonzalez Duque and Alexandra Marquardt have contributed
equally to this work.

<figure><img src="image_3.png" /></figure>

Vanessa Gonzalez Duque
vanessag.duque@tum.de

<footnote n="1">¹ Computer-Aided Medical Procedure and Augmented Reality (CAMP), CIT, Technical University of Munich, Garching bei Muenchen, Germany</footnote>

<footnote n="2">² Munich Center for Machine Learning, Munich, Germany</footnote>

<footnote n="3">³ LS2N Laboratory, Ecole Centrale Nantes, Nantes, France</footnote>

<footnote n="4">⁴ MIP Laboratory, EA 4334, 44000 Nantes, France</footnote>
delineations are drawn along distinct boundaries in clear regions. Subsequently, in regions with less clarity, completion is performed, guided by prior knowledge and experience.

As ultrasound is a dynamic imaging method, it requires anatomical and positional context for accurate localization of structures. Therefore, evaluating segmentation methods, and in particular neural network performance for ultrasound image segmentation, should account for these complexities. As a result, we suggest evaluating separately the ability to delineate the clear regions and the ability to complete the unclear boundary regions.

Despite its importance, most neural networks for ultrasound segmentation are assessed by comparing one expert label with the prediction as a learning signal. Metrics like Dice score or Hausdorff distance rank networks but don't necessarily reflect the network's understanding of border information. Particularly in ultrasound imaging, where borders are often ambiguous, it is important to understand the information that the networks rely on for segmentation to ensure a robust evaluation.

Recent works proposed Grad-CAM [4, 5] and other interpretation methods [6, 7] to understand what networks look at when segmenting medical images. Most of the work mainly focus on MRI or CT and a few on ultrasound [8, 9], evaluating the activation for complete structures.

In this paper, we concentrate on assessing networks based on their learned representation within border regions rather than solely evaluating the accuracy of the predictions. Specifically, we distinguish between distinct and completed boundaries, aiming to emulate the techniques of medical practitioners. In our study, we conduct comprehensive evaluations across four ultrasound datasets for three encoder-decoder style architectures [10–12]. Additionally, we open-source a LEG-3D-US dataset containing segmentation at different depth levels, even in areas where doctors interpolate. Thus, we facilitate the repeatability of the experiments and contribute to annotated medical datasets. In summary, our contribution can be summarized as follows:

1. We present a novel approach for analyzing ultrasound segmentation networks, distinguishing between distinct and completed borders, and mirroring medical practitioners' evaluations.

2. We exploit the Grad-CAM to highlight the specific regions prioritized by each segmentation network across four diverse ultrasound datasets.

3. We make publicly available a LEG-3D-US dataset, facilitating further research and ensuring reproducibility.

# Related work

## Evaluating learned information

While several methods have been developed for enhanced model interpretation [13, 14], Gradient-weighted Class Activation Mapping (Grad-CAM) has emerged as a prominent tool. Initially introduced for classification in [4], Grad-CAM highlights regions of an input image that significantly influence a model's prediction, providing insights into why a model made a specific decision. Many works have employed Grad-CAM to evaluate the learned information of classification networks [4, 5]. When it is evaluated on medical images, its task is to explain the decision-making processes for architectures' comprehension. However, to the best of our knowledge, there are no qualitative Grad-CAM evaluations of ultrasound borders. It is a crucial task to enable reliable diagnoses and effective treatment planning [15], for tumor localization [16], lesion detection [17], or segmentation [18, 19], in a qualitative manner. Hence, in this paper, we aim to analyze network activations for borders, particularly differentiating between distinct and completed types, utilizing Seg-Grad-CAM as an explainability tool.

## Open-source ultrasound datasets

Segmenting ultrasound images using neural networks requires high-quality datasets because of their complexity and difficulty. These datasets, standardized by specific acquisition protocols, should be made openly available to encourage innovation and collaboration within the medical imaging community. Open-source datasets provide a standardized benchmark, enabling researchers globally to train, test, and refine their algorithms, ensuring more rapid advancement of diagnostic tools. Moreover, it democratizes access and allows researchers from varied backgrounds to contribute, leading to diverse and robust models that cover a broader range of medical conditions and patient demographics.

In this section, we briefly describe previously released open-source datasets [20, 21] and one internal 3D prostate dataset. Four ultrasound datasets for thyroid, nerve, leg, and prostate segmentation have been observed in this work. Among them, two have been previously released open-source datasets [20, 21], and one is a private 3D prostate dataset. Additionally, a 3D leg ultrasound dataset (LEG-3D-US) was collected and will be made open-source. Different from the previous datasets, with relatively homogeneous images as the thyroid and prostate datasets, the Leg-3D-dataset con-
tains muscles that are not homogeneous tissues, suffer from blurred and hard-to-delineate borders, and splits segmentation of muscles in 3 groups, which is more challenging compared with a vein vs a muscle segmentation. It could be very interesting to try our method in other more challenging datasets such as kidney, brain, or fetal anatomy, if such datasets were made open-source, for now, we encourage researchers to try our method on their datasets. Following are additional details on the datasets.

**Thyroid dataset:** Kronke et al. [20] provided 32 volumes of 16 participants of the left and the right sides of the neck. With 3 labels segmented: the thyroid, the jugular vein, and the carotid artery. 3D volumes with pixel resolution of 380 × 330 × 300 and voxel spacing of 0.12mm were acquired with a 3D curvilinear probe of 64 channels.

**Nerve-UTP-2D dataset:** Jimenez et al. [21] provides 691 2D ultrasound images of nerves sized 360 × 279 pixels, cropped from original 640 × 480 images, with nerves covering 30% of the area. Captured using a SONOSITE Nano-Max device, it includes images of the sciatic nerve (287 instances), ulnar nerve (221 instances), median nerve (41 instances), and femoral nerve (70 instances). They use a linear 4–16MHz probe.

**Prostate 3D dataset:** It is an internal dataset of 40 patients with suspected cancer, 3D volumes with a pixel resolution of 230 × 230 × 70 and voxel spacing of 0.27 mm. For each participant was performed an ultrasound and a magnetic resonance-T1 (MRI) acquisition. Ultrasound volumes were recorded with a rectal ultrasound probe Wisonic Endokavitär Sonde EV10-4 using the Acuson Juniper Siemens ultrasound machine. Labels of the prostate were transferred from annotations done in the MRI images after registration of the ultrasound volumes.

## Open-source LEG-3D-US dataset

The LEG-3D-US dataset contains 44 ultrasound volumes from legs of participants aged 18–45. It is part of the Achilles tendinopathy study by Crouzier et al. [22], with ethical approval of the local ethics committee (Rennes Ouest V-CPP-MIP-010) with all procedures adhered to the declaration of Helsinki. Images were acquired with the participant’s leg in a custom water tank (Fig. 1a) to avoid muscle deformation due to probe pressure. Freehand ultrasound with six optical tracking camera systems was used (Optitrack, Natural point, USA) for acquiring the 3D volumes. The acquisition took images from knee to ankle, in 4–6 overlapping sweeps (Fig. 1b) using a 50-mm linear probe (4–15 MHz; Aixplorer, Supersonic Imagine, Aix-en-Provence, France). Volumes were created using the compound volume algorithm

in ImFusion Suite software,<fnref n="1" /> with a Gaussian kernel of 5 pixels (Fig. 1c). Each volume consists of a voxel grid with size 564 × 632 × 1443 ± (49 × 38 × 207) and an average spacing 179 of 0.276993 mm³ ± 0.0015 mm³.

Sparse muscle annotations were performed in 2D B-mode high-resolution images on the Stradwin Software [23] by 2 annotators with an intra-operator error of 4%. Annotators segmented gastrocnemius medialis, gastrocnemius lateralis, and soleus muscles. 3D-muscle models were reconstructed with the “ZOI method” [3] and for 15 of the participants, additional refinement was done by one of the experts. We make the LEG-3D-US dataset publicly available.

## Proposed clinician-inspired evaluation

The core of our method is to evaluate segmentation networks using Seg-Grad-CAM in completed and distinct borders. This split is crucial to understanding the expert’s annotation process. Different from CT and MRI which contain regions with similar intensities, ultrasound intensities are not uniform because of the sound’s interactions, making it difficult to separate structures by comparing intensity values among tissues. Instead, doctors rely on anatomical knowledge and prior observations and initially guide themselves based on tissue intersections with a high reflection. Distinct borders can be understood as clear edges in the ultrasound image, while completed borders refer to the doctor’s skill to fill the gaps in order to get anatomically correct smooth structures, looking at previous frames, for example.

We propose to calculate the distinct borders as the Hadamard product between the border label (B, performed by an expert, Fig. 3b) and the edged-smoothed ultrasound image created by applying smoothed and gradient filters to the ultrasound (Fig. 2b). The distinct Border ($B_{distinct}$) can be expressed as:

$$B_{distinct} = \text{Thres}(I * K_{smooth} * K_{sobel}) \odot B \quad (1)$$

Figure 2 presents the challenging ultrasound images of the leg in a cross-sectional view (a) and the extracted edges (b). The obtained distinct borders are in red and the completed in blue (c). We can observe in the 3D view (d) the high probability of getting completed borders in deeper regions where the ultrasound signal is low.

**Pre-evaluated Seg-Grad-CAM:** The main objective of our method is to evaluate the ability of the network to delineate the borders, as physicians do. For this, we visualize the Seg-Grad-CAM of the completed and distinct borders. We

<footnote n="1">¹ ImFusion GmbH, Munich, Germany.</footnote>
<figure><img src="image_4.png" /><figcaption>Fig. 1 LEG-3D-US dataset: a Single ultrasound sweep, b 5 sweeps, c 3D volumes reconstructed, d Sparse annotations e Interpolations f Cross-sectional view soleus (SOL), Gastrocnemius Lateralis (GL), Gastrocnemius Medialis (GM)</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>Fig. 2 Distinct (red) and completed (blue) borders. a Ultrasound image, b Edges on the image, c 2D cross-sectional view with borders, and d 3D view of the borders</figcaption></figure>

observe the activation areas for specific pixels in order to understand the decisions taken by the networks. The core of the Seg-Grad-CAM is the gradient computations. The localization map $L_{\text{Seg-Grad-CAM}}^c$ is computed by first obtaining the gradient with respect to a specific pixel's score class $Y^c$ in the output segmentation map (for $Y^c \in \{B_{\text{distinct}}, B_{\text{completed}}\}$), then computing the global average pooling of these gradients, and finally combining these with the activation maps and applying a ReLU activation. This process is represented by:

$$L_{\text{Seg-Grad-CAM}}^{c} = \text{ReLU}\left(\sum_{k}\left(\frac{1}{w \times h}\sum_{i}\sum_{j}\frac{\partial Y_{i,j}^{c}}{\partial A_{i,j}^{f}}\right)A^{f}\right), \quad (2)$$

where $A^f$ is the activation map of the $f$th feature map from the last convolutional layer $L$ of the segmentation network when processing image $X$, being $\frac{1}{w \times h}$ the global average pooling operator, and $(i, j)$ the spatial coordinates in the feature map. Figure 3 presents the Seg-Grad-CAM for $B_{\text{distinct}}$ (c) and $B_{\text{completed}}$ (d) for just soleus muscle. We can observe that the border segmentation of one muscle takes into consideration the other muscles; this means that there exist a relationship between muscles which provides contextual information for segmentation. When segmenting distinct regions, we can observe an activation mainly in the distinct borders of all the muscles. For completed borders, activations appear in distinct borders of all the muscles, and additional regions like the bones, mimicking doctors' procedure to use key points as reference.
<figure><img src="image_6.png" /><figcaption>**Fig. 3** Seg-Grad-CAM activations for UNet for just the soleus muscle after split into distinct and completed borders: Labels correspond to purple: soleus, green: Gastrocnemius Lateralis, and pink: Gastrocnemius Medialis</figcaption></figure>

## Experiments and results

In order to evaluate the accuracy of the networks to predict distinct and completed borders on ultrasound datasets, we train 3 different encoder-decoder architectures, using Monai framework [24] (an open-source Pytorch-based framework designed for medical imaging research), on 4 different datasets, for 3 different cross-validation runs. Losses like Dice, cross-entropy, and Dice-Cross-entropy were evaluated. To solve the imbalance problem in the thyroid and the leg dataset, the weighting of cross-entropy was calculated with the inverse of the percentage of pixels belonging to that class in the training set and normalization of the class weights to ensure they all sum up to 1. For our experiments, all datasets were split patient-wise in 70-20-10% for training, validation, and testing, respectively. Data augmentation techniques like flipping the images in the vertical axis and min-max histogram normalization were applied to the images. The network trained until convergence with batch size to 2. We did not use pre-trained weights due to the network's adaptation on big size volumes, but in custom smaller datasets, we use that pre-trained weights can improve the network's performance. We used Adam optimizer with a learning rate of 0.001 and a StepLR scheduler, with a learning rate decay frequency set at intervals of 10 epochs, coupled with a decay factor of 0.5. We evaluate the mean and standard deviation of different metrics, including Dice, Hausdorff distance 95% (HD95) and the normalized surface distance (NSD), for the images in the test dataset.

## Common border segmentation metrics evaluation

We calculate different segmentation metrics, including the normalized surface distance (NSD) that specifically evaluates the focus on the borders. We report the results in Table 1. Our findings indicate that the UNet transformer architecture con-

sistently displayed inferior performance across all datasets compared to the other two network architectures, likely due to the number of data transformers' architectures needed. It showed an average performance gap of 6% in 2D versus 27% in 3D. Selecting the best architecture between UNet and A-UNet was not straightforward due to their closer metric values, especially for the thyroid and prostate datasets. We performed additional evaluations to understand the dispersion of the metrics for each slice, by computing violin plots available in the supplementary material. We observe that the statistical difference depends on the metric used. In the prostate dataset, using the Dice metric does not present a difference between UNet and A-UNet, while using the NSD score presents a difference. The selection of the metric is important and depends on the modality and the desired clinical use.

Further, we compute the Seg-Grad-CAM of the background label in order to understand how the network delineates the borders of desired structures, see Fig. 4. To understand the influence of different attention modules on different architectures, we visualized the focus areas of the networks. In the case of UNet, Seg-Grad-CAM has wider spread attention over the image, while Attention UNet uses the attention modules to focus on more specific areas, and mainly the borders. Also, in the case of UNetTR, patch-wise processing of the image results in patch-wise Seg-Grad-CAM patterns. In fact, since ultrasound has no well-defined borders, the self-attention maintains active ineffective global correlations between patches. It can be observed that networks mainly focus on structures with high reflection, such as tissue interfaces, and depending on the network the attention area of the background gets slightly more dispersed or focused. However, it doesn't focus on any specific anatomical points, and therefore, it does not provide sufficient information for analyzing the networks.
<table><caption>**Table 1** Evaluated metrics with mean and standard deviation of the test datasets for 2D and 3D architectures</caption><thead><tr><th rowspan="2">Dataset<br>Metrics</th><th colspan="3">Leg-3D-US</th><th colspan="3">Nerve-UTP-2D</th></tr><tr><th>Dice</th><th>HD95</th><th>NSD</th><th>Dice</th><th>HD95</th><th>NSD</th></tr></thead><tbody><tr><td>UNet</td><td><b>0.789 ± 0.029</b></td><td><b>5.61 ± 1.62</b></td><td><b>0.962 ± 0.023</b></td><td><b>0.739 ± 0.283</b></td><td><b>11.28 ± 15.62</b></td><td><b>0.815 ± 0.319</b></td></tr><tr><td>A-UNet</td><td>0.587 ± 0.048</td><td>8.29 ± 1.58</td><td>0.866 ± 0.036</td><td>0.73 ± 0.298</td><td>11.72 ± 11.07</td><td>0.791 ± 0.316</td></tr><tr><td>UNeTR</td><td>0.519 ± 0.161</td><td>14.75 ± 4.12</td><td>0.757 ± 0.107</td><td>0.661 ± 0.249</td><td>18.06 ± 18.56</td><td>0.719 ± 0.258</td></tr></tbody></table><table><thead><tr><th rowspan="2">Dataset<br>Metrics</th><th colspan="3">Thyroid</th><th colspan="3">Prostate</th></tr><tr><th>Dice</th><th>HD95</th><th>NSD</th><th>Dice</th><th>HD95</th><th>NSD</th></tr></thead><tbody><tr><td>UNet</td><td>0.888 ± 0.053</td><td><b>5.12 ± 2.71</b></td><td>0.966 ± 0.018</td><td><b>0.816 ± 0.075</b></td><td>6.84 ± 4.02</td><td><b>0.952 ± 0.048</b></td></tr><tr><td>A-UNet</td><td><b>0.892 ± 0.033</b></td><td>6.81 ± 1.67</td><td><b>0.972 ± 0.014</b></td><td>0.815 ± 0.092</td><td><b>6.71 ± 4.05</b></td><td>0.947 ± 0.065</td></tr><tr><td>UNeTR</td><td>0.636 ± 0.105</td><td>27.20 ± 8.12</td><td>0.766 ± 0.034</td><td>0.523 ± 0.179</td><td>10.66 ± 6.28</td><td>0.77 ± 0.217</td></tr></tbody></table>

With HD95 being the Hausdorff distance and NSD the normalize surface distance. In bold is highlighted the best performance

<figure><img src="image_7.png" /><figcaption>**Fig. 4** 3D Seg-Grad-CAM evaluation of the background label with predictions border overlapped in white</figcaption></figure>

## Evaluation of network accuracy for predicting distinct and completed borders

In our approach, we evaluate instead the image areas activated using Seg-Grad-CAM for distinct and completed borders (see Fig. 5). We observe that the attention areas are different depending on the network architectures. In general, the distinct borders get activated in interfaces with high reflection. In the case of leg muscle segmentation, there is a high attention on the fat layer outlining the upper part of the muscle, resulting in a clearer border.

In the case of completed borders, predictions get activated depending on the network attention method. For instance, UNetR divides the volume into patches, which worsens the completion of areas. UNet, on the other hand, handles features at different scales with the encoder, allowing it to take into account distinct borders when predicting completed ones. Additionally, we observe that other structures or organs, such as bones or trachea, were used to provide more context and improve the completed borders. This is similar to the

way doctors look for key points to help them segment borders. We hypothesize that the striping artifacts could be due to the observation of the comet-tail artifact generated by the vein in the thyroid dataset or due to the compounding lines created when compounding several 2D B-mode images in the leg dataset. However, more research should be done in this direction.

In a quantitative evaluation of distinct and completed prediction, we extract the borders of the ground truth labels and use this as a reference border. To obtain the percentage of distinct borders, we first calculate the Hadamard product, see Sect. 4 and then multiply it with the reference border. The completed border is the complementary of distinct, both summing 100% for the reference segmentation.

Table 2 provides an analysis of the leg and thyroid ultrasound datasets. For both datasets, the first row indicates the reference border percentage. The subsequent rows showcase predictions from three different segmentation models (UNet, A-UNet, and UNetR) in comparison to the total reference label border. Each model's predictions are divided
<figure><img src="image_8.png" /><figcaption>**Fig. 5** Anatomical networks observations: On the thyroid, we can identify the Trachea, while on the leg other structures like the Fat layer and the Fibula are also activated</figcaption></figure>

<table><caption>**Table 2** True-positive percentage in the test dataset predicted with respect to the complete border</caption><thead><tr><th rowspan="2"></th><th colspan="3">Leg-3D-US</th><th colspan="3">Thyroid</th></tr><tr><th>B<sub>TP</sub></th><th>B<sub>Distinct</sub></th><th>B<sub>Completed</sub></th><th>B<sub>TP</sub></th><th>B<sub>Distinct</sub></th><th>B<sub>Completed</sub></th></tr></thead><tbody><tr><td>Reference<sub>Border</sub> [%]</td><td>100</td><td>54.8 ± 7.6</td><td>45.2 ± 4.3</td><td>100</td><td>86.7 ± 4.8</td><td>13.3 ± 4.8</td></tr><tr><td>UNet<sub>Predictions</sub></td><td>82.1 ± 3.3</td><td>42.2 ± 1.9</td><td>39.9 ± 2.1</td><td>86.3 ± 1.5</td><td>80.4 ± 2.7</td><td>5.9 ± 1.4</td></tr><tr><td>A – UNet<sub>Predictions</sub></td><td>75.5 ± 7.4</td><td>50.4 ± 2.4</td><td>35.1 ± 2.8</td><td>88.4 ± 2.3</td><td>83.0 ± 1.2</td><td>5.1 ± 1.3</td></tr><tr><td>UNetR<sub>Predictions</sub></td><td>37.2 ± 3.6</td><td>23.6 ± 2.5</td><td>13.6 ± 1.3</td><td>72.8 ± 7.9</td><td>70.5 ± 3.3</td><td>4.3 ± 2.1</td></tr></tbody></table>

Reference expresses the percentage of distinct and completed borders with respect to the total border

into three categories: total true positive ($B_{TP}$), distinct borders ($B_{Distinct}$), and completed borders ($B_{Completed}$), with the percentages and standard deviations provided. A high percentage for the prediction of distinct borders for all tasks can be observed, showing that the networks perform better for this task.

## Discussion and perspectives

In this work, we focus on analyzing ultrasound segmentation methods on their ability to learn boundaries between organs or structures. Different from MRI and CT scans, where similar tissues typically exhibit uniform values; in ultrasounds, intensities in the same tissue can fluctuate. We introduce a clinically inspired analysis that emphasizes border differentiation, categorizing them into distinct and completed borders. Seg-Grad-CAM is primarily employed in our study to enhance model interpretability and examine the network's focal points but is not the central focus. Our findings reveal that these networks, much like medical professionals, adjust their focus depending on border clarity. The completion of borders takes into consideration both the clear borders and key anatomical points, alongside an intrinsically learned shape model.

However, it is important to clarify that Grad-CAM is subject to limitations that may lead to oversimplification, although it is widely used for its ability to visualize deep

neural network decisions. Our experiment in supplementary material, including the evaluation of “signal lost patches” for the leg dataset and the impact of noise and contrast artifacts, evaluates the Grad-CAM’s robustness and its capacity to ignore non-informative areas while effectively highlighting relevant features. Our evaluation in distinct and completed borders is independent of the explainability method; future works could explore the advantages of using other explainability methods more robust to noise.

A further contribution is the provision of an open-access 3D leg ultrasound dataset. Public labeled datasets add value to the biomedical community helping in advancing of medical diagnostics. The leg dataset depicts the anatomical intricacies of the lower limb musculature, which can be useful in improving therapeutic interventions, particularly in the context of muscular pathologies like Duchenne muscular dystrophy. Furthermore, the dataset’s utility extends to the field of sports medicine, where it can serve to analyze the relationship between an athlete’s performance and the dynamic evolution of leg muscles, enabling the refinement of personalized training regimens. Importantly, this initiative circumvents the constraints posed by conventional imaging modalities such as MRI and CT, offering a non-invasive, cost-effective, and repeatedly applicable alternative.

For quantitative evaluation, we calculate separately the performance of the network to learn distinct borders or more complex ones (completed). Such metric splitting can be done for other metrics like IoU or normalized surface distance to
get an idea of individual activation depending on the com-
plexity of the task. Future perspectives can explore how
anatomical shape models are learned in the networks and
boost their performance using the split in distinct and com-
pleted boundaries in the loss.

**Supplementary Information** The online version contains supplementary material available at https://doi.org/10.1007/s11548-024-03170-7.

**Funding** Open Access funding enabled and organized by Projekt DEAL. This work has been supported in part by the European Regional Development Fund, the Pays de la Loire region on the Connect Talent scheme (MILCOM Project) and Nantes Metropole (Convention 2017-10470).

**Data availability** Data, code and/or material availability. Available at https://github.com/A13xand1a/segmentation-border-analysis.

Declarations

**Conflict of interest** The authors have no relevant financial or non-financial interests to disclose.

**Ethics approval** The experimental procedures were approved by the local ethics committee (Rennes Ouest V, CPP-MIP-010), and all procedures adhered to the Declaration of Helsinki.

Consent to participate and publish Not applicable.

**Open Access** This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

References

1. Oh YS, Early DS, Azar RR (2005) Clinical applications of endoscopic ultrasound to oncology. Oncology 68(4-6):526-537

2. Noble JA, Boukerroui D (2006) Ultrasound image segmentation: a survey. IEEE TMI 25(8):987-1010. https://doi.org/10.1109/TMI.2006.877092

3. Duque VG, Alchanti D, Crouzier M, Nordez A, Lacourpaille L, Mateus D (2020) Low-limb muscles segmentation in 3d freehand ultrasound using non-learning methods and label transfer. In: 16th SIPAIM, vol 11583, pp 154-163

4. Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D (2017) Grad-cam: visual explanations from deep networks via gradient-based localization. In: Proceedings of the IEEE ICCV conference, pp 618-626

5. Zhou B, Khosla A, Lapedriza A, Oliva A, Torralba A (2016) Learning deep features for discriminative localization. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2921-2929

6. Adebayo J, Gilmer J, Muelly M, Goodfellow I, Hardt M, Kim B (2018) Sanity checks for saliency maps. Adv Neural Inf Process Syst 31:9505

7. Zhang Y, Khakzar A, Li Y, Farshad A, Kim ST, Navab N (2021) Fine-grained neural network explanation by identifying input features with predictive information. Adv Neural Inf Process Syst 34:20040-20051

8. Jung Y, Kim T, Han M-R, Kim S, Kim G, Lee S, Choi YJ (2022) Ovarian tumor diagnosis using deep convolutional neural networks and a denoising convolutional autoencoder. Sci Rep 12(1):17024

9. Du R, Chen Y, Li T, Shi L, Fei Z, Li Y (2022) Discrimination of breast cancer based on ultrasound images and convolutional neural network. J Oncol 8:32

10. Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In: Medical image computing and computer-assisted intervention—18th MICCAI conference. Springer, pp 234–241

11. Oktay O, Schlemper J, Folgoc LL, Lee M, Heinrich M, Misawa K, Mori K, McDonagh S, Hammerla NY, Kainz B, Glocker B, Rueckert D (2018) Attention u-net: learning where to look for the pancreas. MIDL conference

12. Hatamizadeh A, Tang Y, Nath V, Yang D, Myronenko A, Landman B, Roth HR, Xu D (2022) Unetr: transformers for 3d medical image segmentation. In: Proceedings of the IEEE/CVF conference, pp 574–584

13. Draelos RL, Carin L (2022) Explainable multiple abnormality classification of chest CT volumes. Artif Intell Med 132:102372

14. Jamil MS, Banik SP, Rahaman GA, Saha S (2023) Advanced grad-cam++: improved visual explanations of CNN decisions in diabetic retinopathy. In: Computer vision and image analysis for industry 4.0, pp 64–75

15. Zhang H, Hu L, Liang W, Li Z, Yuan M, Ye Y, Wang Z, Ren Y, Li X (2023) BCT-OFD: bridging CNN and transformer via online feature distillation for covid-19 image recognition. Int J Mach Learn Cybern 6:1–20

16. Lee S, Lee J, Lee J, Park C-K, Yoon S (2018) Robust tumor localization with pyramid grad-cam. CoRR

17. Jiang H, Xu J, Shi R, Yang K, Zhang D, Gao M, Ma H, Qian W (2020) A multi-label deep learning model with interpretable grad-cam for diabetic retinopathy classification. In: 2020 42nd Annual international conference of the IEEE engineering in medicine and biology society (EMBC). IEEE, pp 1560–1563

18. Xiao M, Zhang L, Shi W, Liu J, He W, Jiang Z (2021) A visualization method based on the grad-cam for medical image segmentation model. In: 2021 EIECS conference. IEEE, pp 242–247

19. Hasany SN, Petitjean C, Mériaudeau F (2023) SEG-XRES-CAM: explaining spatially local regions in image segmentation. In: Proceedings of the IEEE/CVF CVPR conference, pp 3732–3737

20. Kronke M, Eilers C, Dimova D, Kohler M, Buschner G, Schweiger L, Konstantinidou L, Makowski M, Nagarajah J, Navab N, Weber W, Wendler T (2022) Tracked 3d ultrasound and deep neural network-based thyroid segmentation reduce interobserver variability in thyroid volumetry. Plos One 17(7):0268550

21. Jimenez-Castaño CA, Álvarez-Meza AM, Aguirre-Ospina OD, Cárdenas-Peña DA, Orozco-Gutiérrez ÁA (2021) Random Fourier features-based deep learning improvement with class activation interpretability for nerve structure segmentation. Sensors 21(22):7741

22. Crouzier M, Lacourpaille L, Nordez A, Tucker K, Hug F (2018) Neuromechanical coupling within the human triceps surae and its consequence on individual force-sharing strategies. J Exp Biol 221(21):187260

23. Gee A, Prager R, Treece G, Cash C, Berman L (2004) Processing and visualizing three-dimensional ultrasound data. Br J Radiol 77(2):186-193
24. Cardoso MJ, Li W, Brown R, Ma N, Kerfoot E, Wang Y, Murrey B, Myronenko A, Zhao C, Yang D, Feng A (2022) Monai: an open-source framework for deep learning in healthcare. arXiv preprint arXiv:2211.02701

**Publisher's Note** Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.