# Multi-Features-Based Automated Breast Tumor Diagnosis Using Ultrasound Image and Support Vector Machine

Authors: Zhemin Zhuang; Zengbiao Yang; Shuxin Zhuang; Alex Noel Joseph Raj; Ye Yuan; Ruban Nersisson
Journal: Computational Intelligence and Neuroscience
 (2021)

---

## Abstract

Breast ultrasound examination is a routine, fast, and safe method for clinical diagnosis of breast tumors. In this paper, a classification method based on multi-features and support vector machines was proposed for breast tumor diagnosis. Multi-features are composed of characteristic features and deep learning features of breast tumor images. Initially, an improved level set algorithm was used to segment the lesion in breast ultrasound images, which provided an accurate calculation of characteristic features, such as orientation, edge indistinctness, characteristics of posterior shadowing region, and shape complexity. Simultaneously, we used transfer learning to construct a pretrained model as a feature extractor to extract the deep learning features of breast ultrasound images. Finally, the multi-features were fused and fed to support vector machine for the further classification of breast ultrasound images. The proposed model, when tested on unknown samples, provided a classification accuracy of 92.5% for cancerous and noncancerous tumors.


---

## Body

## 1. Introduction

International Agency for Research on Cancer (IARC) reported that breast cancer accounts for about 24.2% of cancers diagnosed in women worldwide [1]. It is also the leading fatal cause in women, accounting for about 15%. With the development of modern medicine, if breast cancer is diagnosed early, the survival rate of patients will be significantly improved [2]. Breast tumors are usually examined by computerized tomography (CT), magnetic resonance imaging (MRI), molybdenum target X-ray, far infrared, ultrasound (US), and other methods. Among them, the US has become the preferred choice for early breast cancer screening due to its cost-effectiveness and more robust images [3]. However, the Breast Imaging-Reporting and Data System (BI-RADS) [4] grades diagnosed by different clinicians for the same patient are subjective and different since some features in the breast ultrasound (BUS) images are not typically visible to diagnose [5]. Besides, different breast lesions show different features in BUS images. Also, experience and the ability to understand the visual clues from BUS images are essential in reducing false negative detection. The count shows that the missed diagnosis of medical imaging in disease diagnosis can be between 10% and 30% [6].Artificial intelligence (AI) can assist doctors in making a more accurate judgment because of its objectivity and versatility. AI diagnosis of benign and malignant BUS images can be divided into deep learning and feature extraction. Deep learning transforms the raw BUS images into much higher-dimension expression through convolutional neural networks (CNNs). Multi-level and multi-aspect features are obtained by training the network model, which makes the obtained features have a more robust generalization and expression. Deep learning is often used in the automatic classification of BUS images. For example, in reference [7], 166 malignant and 292 benign BUS images were trained and classified by using a neural network composed of three convolution layers and two fully connected layers. Qi et al. [3] used Mt-Net (malignant tumors) and Sn-Net (solid nodules) to classify BUS images, where Mt-Net was used to detect malignant tumors and Sn-Net was used to detect solid nodules. Although deep learning has achieved good results, they are constrained by the need for a higher number of ground truth (GT).Feature extraction techniques identify useful characteristic features (CFs) from the original images, where the original image is transformed into a group of features with obvious physical significance, to achieve the purpose of dimensionality reduction. For example, in reference [8], the region growing method was used to segment the lesion, and the histogram method was used to calculate six histograms from the posterior shadowing region (PS). Finally, BUS images were divided into PS enhancement and PS nonenhancement by using the six histograms and multilayer perceptron (MLP). However, PS is only one of the features to judge the benign and malignant BUS images and lacks accuracy to make a classification.In this paper, (1) by analyzing the different manifestations of benign and malignant breast tumors in ultrasound images, combined with the clinical experience of experts, different and effective characteristic features were designed manually. (2) In order to assist the classification of breast tumor ultrasound images, we used transfer learning for extracting deep learning features. Finally, (3) SVM was used to integrate characteristic features and deep learning features and present an effective classification.


## 2. Materials and Methods

Since benign and malignant breast BUS images have different histological structures and biological characteristics, they exhibit different properties on BUS images. The malignant tumors such as ductal carcinoma in situ [9], due to their invasive nature, penetrate through the ducts and into adjacent fibrous and adipose tissues. It forms a blurred mixed zone between the tumor and the tissues and complex edge. Besides, the complex interstitial components and hyaline transformation of malignant lesions often lead to scattering of acoustic signals [10, 11]. During the decision process, the specialists often consider the orientation, posterior shadowing (PS), edge indistinctness (EI), and shape complexity (SC) of the tumors as essential features to identify them as benign or malignant. These characteristic features (CFs) of BUS images are not only a critical judgment for clinical diagnosis but also a significant basis for BUS image classification based on feature extraction. Hence, we proposed a method to classify breast lesions using multi-features (MFs) and SVM. The proposed method firstly employed a level set technique to segment the tumor region of BUS images. From the contour of the segmented tumor, orientation and edge indistinctness (EI) scores were calculated. Next, using Hu moments, we determined the characteristic of the posterior shadowing (PS) region. Later, the fractal divider method was used to calculate the shape complexity (SC) score of tumor contour. Meanwhile, the pretrained VGG16 model was used as a feature extractor to get deep learning features (DLFs) of BUS images. Finally, we classified the BUS images based on the multi-features (MFs) obtained above and SVM. The above process is illustrated in Figure 1.Figure 1
The proposed methodology.


## 3. Lesion Segementation and Feature Calculation

Due to the complexity involved during the ultrasound examination, the acquired images contain speckle noise, image artifacts, and weak boundaries that hurt the segmentation process. Accurate segmentation can effectively improve the accuracy of classification [12]. Therefore, for accurate extraction of tumor regions, conventional segmentation techniques may not provide desired results. Literature suggests that level set techniques are useful for segmentation problems related to topological changes, and hence we used an advanced level set segmentation algorithm based on geometric active contour model and curve evolution theory to complete the lesion segmentation of BUS images [13]. The technique employed an iterative method to segment the tumor region within the BUS image accurately. The following paragraph briefly explains the use of a level set algorithm to segment the BUS images.The level set algorithm that we used in this paper does not depend on the gradient information of the BUS image. Therefore, it is insensitive to noise and has a significant advantage in medical image processing [14, 15]. Here we employed the Distance Regularized Level Set Evolution (DRLSE) [16] model that eliminates the need for reinitialization but employs a distance regularization term and energy functions to propagate the zero-level set function (LSF) towards the desired locations. The energy function Eϕ can be defined as follows.Forϕ:Ω⟶ℜ,(1)Eϕ=μ∫Ωp∇ϕdx+λ∫Ωgδϕ∇ϕdx+α∫ΩgH−ϕdx,where ∫Ωp∇ϕdx is the regularization term and ∫Ωgδϕ∇ϕdx and ∫ΩgH−ϕdx are the external energy terms. μ>0, λ>0, α∈ℜ are the coefficients, respectively, H and δ are the Heaviside function and Dirac delta function, respectively, and p is a potential function.Due to the addition of the distance regularization termp∇ϕ in equation (1), the deviation between the level set and the signed distance function (SDF) is automatically corrected in each iteration of the level set function, thus maintaining stability and avoiding reinitialization.The following gradient descent flow function can be obtained by differentiating equation (1), to realize the extraction of the tumor region in BUS image while minimizing the energy function.(2)∂ϕ∂t=μΔϕ−div∇ϕ∇ϕ+λδεϕdivg∇ϕ∇ϕ+αgδεϕ.The implementation of DRLSE for an application is based on the flowchart illustrated in Figure2, which includes (a) initialization of LSF ϕ0 and narrowband B0; (b) updating the LSF and narrowband region; (c) updating the pixel values on the narrowband based on ϕi,jk+1, where i,j is called zero-crossing point, k is the number of iteration, and τ is the time step; and (d) termination of the process, if the prescribed number of consecutive zero-crossing has opposite signs or the expected number iterations is reached. The segmentation results of the lesions in breast ultrasound images are shown in Figure 3.Figure 2
The evolution process of DRLSE.Figure 3
Segmentation results from the DRLSE model: (a), (b), and (c) correspond to zero LSF. (a-1) to (a-3), (b-1) to (b-3), and (c-1) to (c-3) illustrate the evolution of the LSF at 200, 600, and 800 iterations, respectively.


### 3.1. Computation of Orientation

The growth characteristics of benign and malignant tumors vary and therefore show different orientations. Here we used contour obtained from the segmentation process to facilitate the calculation of tumor orientation. First, we transversed the segmented contour in both horizontal and vertical directions and obtain four points: topxu,yu, bottom xd,yd, leftmost xl,yl, and rightmost xr,yr. These points are vertices obtained from the intersection of lines lu,ld,ll, and lr with upper, lower, leftmost, and rightmost extreme regions of contour, respectively, as shown in Figure 4.Figure 4
The contour of the segmented tumor region of the BUS image.xu,yu, xd,yd, xl,yl, and xr,yrare the four vertices obtained from the intersection of lines lu,ld, ll, and lr with the upper, lower, leftmost, and rightmost extreme of contour, respectively.Next, using equation (3), we computed the tumor orientation, which is expressed as a ratio between the height to the width of the tumor.(3)Orientation=yu−ydxr−xl.


### 3.2. Computation of Edge Indistinctness (EI) Score of Lesions

Commonly malignant tumors penetrate deeper into the tissues causing indistinctive margins that are different from the benign ones. Therefore, to measure the edge indistinctness more comprehensively, we extracted a region ofm x n pixels from the top and bottom vertices of the contour, i.e., around xu,yu and xd,yd points, as shown in Figure 5.Figure 5
(a) and (b) illustrate segmented contour shown in green and selectedn × m regions around xu,yu and xd,yd points in white, respectively. Here we set n = 15 and m = 21.
(a)(b)For the extractedm × n region, we separately calculated the standard deviation along the row and the column as given in equations (4) and (5). We defined EI score (equation (6)) as the maximum of the standard deviation computed along with the row and column directions:(4)xstd=∑i=1nxstd_in,(5)ystd=∑j=1mystd_jm,(6)EI score=maxxstd,ystd.For our experiments, we have selected two regions aroundxu,yu and xd,yd points and accordingly computed two EI scores: EIscoreup and EIscoredown, respectively.In cancer diagnosis, the edge strength (blurring) is an important index that is used to classify the tumor. However, there are differences in the degree of blurring across different sections along the tumor boundary. Thus, we defined the average values ofEIscoreup and EIscoredown as EI score, as shown in the following equation:(7)EI score=EIscoreup+EIscoredown2.


### 3.3. Computation of the PS Score of Posterior Shadowing Region by Using Hu Moments

The texture characteristics of the PS region are generally different for benign and malignant tumors. Literature suggests that moments can be used for analyzing texture characteristics [17], and therefore, we used moment invariants proposed by Hu [18] to compare the PS regions of different BUS images quantitatively. The PS region of the BUS image was extracted based on the bottom xd,yd, the leftmost xl,yl, and the rightmost xr,yr points of the contour, as shown in Figure 6.Figure 6
The PS regions. (a) and (b) are the benign and malignant BUS images, respectively. The red line illustrates the segmented tumor contour; the white region represents the PS region considered for PS score computation.
(a)(b)Letfx,y be the extracted PS region; then, its p+q order of the geometric moment can be defined as follows:(8)mpq=∑y=1N∑x=1Mxpyqfx,y,p,q=0,1,2….Their central moments can be defined as(9)μpq=∑y=1N∑x=1Mx−x¯py−y¯qfx,y,p,q=0,1,2…,where x¯ and y¯ are the center of gravity coordinates of the image, given by(10)x¯=m10m00,y¯=m01m00.The normalized central moment is defined as(11)ηpq=μpqμ00ρ,where ρ=p+q/2+1.Normally, seven Hu moments can be computed using the second- and third-order normalized central moments. But here we only use the first moment as it is sufficient to provide a score that could differentiate PS regions of different BUS images effectively. Accordingly, we substitutedp = 0 and q = 2 in equation (11) and defined the PS score as(12)PS score=η20+η02.To have a clear distinction, the PS score is transformed as(13)PS score=−signPS score∗log10absPS score.


### 3.4. Computation of Shape Complexity (SC) Score Based on Fractal Dimensions

The shape is one of the critical factors clinical experts use to classify tumors as benign or malignant. Cancerous tumors have complex contours, whereas benign ones have simpler structures. Therefore, we proposed a technique based on fractals to quantify the shape complexity of the segmented tumors. In image processing, fractals have been widely used in US image segmentation. Omiotek et al. [19] used fractals to quantify the texture of thyroid US images. Lin et al. [20] used fractals to determine the area of alveolar bone defect, and recently Zhuang et al. [21] collectively used fuzzy enhancement and fractal length to segment the US image of atherosclerosis successfully. Also, the fractal theory was successfully used to measure the irregular coastlines and the fault geometries propagated by earthquakes. For example, Mandelbrot [22] employed power law to relate the costal length to the different linear rulers, and Okubo and Aki [23] quantified complex fault geometries to large values of fractal dimension.Here the divider method used by cartographers [23–25] was employed to quantify the shape complexity of the segmented tumor. To measure the complexity, we drew circles of different radii along the boundary of the segmented contour. The starting point was chosen as the top point xu,yu and the circles of different radii were drawn dividing the contours, as shown in Figure 7. Let N (R) be the number of the circles and R be their corresponding radius; then, according to [26, 27], we can relate N (R) and R as(14)Log10NR=a+bLog10R,where “a” and “b” are constants that are obtained through least-squares fit between Log10NR and Log10R and “b” is the slope of the line that represents the fractal dimension, which determines the SC score of the contour.Figure 7
Illustration of divider method. (a)–(h) show the circles drawn with different radii along the contour of the segmentation tumor. (a)R = 4. (b) R = 5. (c) R = 6. (d) R = 7. (e) R = 8. (f) R = 9. (g) R = 10. (h) R = 11.
(a)(b)(c)(d)(e)(f)(g)(h)The illustration of the divider method for a sample BUS image is shown in Figure7, and Table 1 presents R and their corresponding N (R) values. Further Figure 8 demonstrates the SC score obtained by the least-square fitting of N (R) and R for the sample BUS image shown in Figure 7.Table 1
Number of circlesN (R) and their radius R for BUS image shown in Figure 7.
Figure7(a)(b)(c)(d)(e)(f)(g)(h)R4567891011N (R)10985726153464337Figure 8
Computation of SC score for the measurements recorded in Table1 on equation (15).


## 3.1. Computation of Orientation

The growth characteristics of benign and malignant tumors vary and therefore show different orientations. Here we used contour obtained from the segmentation process to facilitate the calculation of tumor orientation. First, we transversed the segmented contour in both horizontal and vertical directions and obtain four points: topxu,yu, bottom xd,yd, leftmost xl,yl, and rightmost xr,yr. These points are vertices obtained from the intersection of lines lu,ld,ll, and lr with upper, lower, leftmost, and rightmost extreme regions of contour, respectively, as shown in Figure 4.Figure 4
The contour of the segmented tumor region of the BUS image.xu,yu, xd,yd, xl,yl, and xr,yrare the four vertices obtained from the intersection of lines lu,ld, ll, and lr with the upper, lower, leftmost, and rightmost extreme of contour, respectively.Next, using equation (3), we computed the tumor orientation, which is expressed as a ratio between the height to the width of the tumor.(3)Orientation=yu−ydxr−xl.


## 3.2. Computation of Edge Indistinctness (EI) Score of Lesions

Commonly malignant tumors penetrate deeper into the tissues causing indistinctive margins that are different from the benign ones. Therefore, to measure the edge indistinctness more comprehensively, we extracted a region ofm x n pixels from the top and bottom vertices of the contour, i.e., around xu,yu and xd,yd points, as shown in Figure 5.Figure 5
(a) and (b) illustrate segmented contour shown in green and selectedn × m regions around xu,yu and xd,yd points in white, respectively. Here we set n = 15 and m = 21.
(a)(b)For the extractedm × n region, we separately calculated the standard deviation along the row and the column as given in equations (4) and (5). We defined EI score (equation (6)) as the maximum of the standard deviation computed along with the row and column directions:(4)xstd=∑i=1nxstd_in,(5)ystd=∑j=1mystd_jm,(6)EI score=maxxstd,ystd.For our experiments, we have selected two regions aroundxu,yu and xd,yd points and accordingly computed two EI scores: EIscoreup and EIscoredown, respectively.In cancer diagnosis, the edge strength (blurring) is an important index that is used to classify the tumor. However, there are differences in the degree of blurring across different sections along the tumor boundary. Thus, we defined the average values ofEIscoreup and EIscoredown as EI score, as shown in the following equation:(7)EI score=EIscoreup+EIscoredown2.


## 3.3. Computation of the PS Score of Posterior Shadowing Region by Using Hu Moments

The texture characteristics of the PS region are generally different for benign and malignant tumors. Literature suggests that moments can be used for analyzing texture characteristics [17], and therefore, we used moment invariants proposed by Hu [18] to compare the PS regions of different BUS images quantitatively. The PS region of the BUS image was extracted based on the bottom xd,yd, the leftmost xl,yl, and the rightmost xr,yr points of the contour, as shown in Figure 6.Figure 6
The PS regions. (a) and (b) are the benign and malignant BUS images, respectively. The red line illustrates the segmented tumor contour; the white region represents the PS region considered for PS score computation.
(a)(b)Letfx,y be the extracted PS region; then, its p+q order of the geometric moment can be defined as follows:(8)mpq=∑y=1N∑x=1Mxpyqfx,y,p,q=0,1,2….Their central moments can be defined as(9)μpq=∑y=1N∑x=1Mx−x¯py−y¯qfx,y,p,q=0,1,2…,where x¯ and y¯ are the center of gravity coordinates of the image, given by(10)x¯=m10m00,y¯=m01m00.The normalized central moment is defined as(11)ηpq=μpqμ00ρ,where ρ=p+q/2+1.Normally, seven Hu moments can be computed using the second- and third-order normalized central moments. But here we only use the first moment as it is sufficient to provide a score that could differentiate PS regions of different BUS images effectively. Accordingly, we substitutedp = 0 and q = 2 in equation (11) and defined the PS score as(12)PS score=η20+η02.To have a clear distinction, the PS score is transformed as(13)PS score=−signPS score∗log10absPS score.


## 3.4. Computation of Shape Complexity (SC) Score Based on Fractal Dimensions

The shape is one of the critical factors clinical experts use to classify tumors as benign or malignant. Cancerous tumors have complex contours, whereas benign ones have simpler structures. Therefore, we proposed a technique based on fractals to quantify the shape complexity of the segmented tumors. In image processing, fractals have been widely used in US image segmentation. Omiotek et al. [19] used fractals to quantify the texture of thyroid US images. Lin et al. [20] used fractals to determine the area of alveolar bone defect, and recently Zhuang et al. [21] collectively used fuzzy enhancement and fractal length to segment the US image of atherosclerosis successfully. Also, the fractal theory was successfully used to measure the irregular coastlines and the fault geometries propagated by earthquakes. For example, Mandelbrot [22] employed power law to relate the costal length to the different linear rulers, and Okubo and Aki [23] quantified complex fault geometries to large values of fractal dimension.Here the divider method used by cartographers [23–25] was employed to quantify the shape complexity of the segmented tumor. To measure the complexity, we drew circles of different radii along the boundary of the segmented contour. The starting point was chosen as the top point xu,yu and the circles of different radii were drawn dividing the contours, as shown in Figure 7. Let N (R) be the number of the circles and R be their corresponding radius; then, according to [26, 27], we can relate N (R) and R as(14)Log10NR=a+bLog10R,where “a” and “b” are constants that are obtained through least-squares fit between Log10NR and Log10R and “b” is the slope of the line that represents the fractal dimension, which determines the SC score of the contour.Figure 7
Illustration of divider method. (a)–(h) show the circles drawn with different radii along the contour of the segmentation tumor. (a)R = 4. (b) R = 5. (c) R = 6. (d) R = 7. (e) R = 8. (f) R = 9. (g) R = 10. (h) R = 11.
(a)(b)(c)(d)(e)(f)(g)(h)The illustration of the divider method for a sample BUS image is shown in Figure7, and Table 1 presents R and their corresponding N (R) values. Further Figure 8 demonstrates the SC score obtained by the least-square fitting of N (R) and R for the sample BUS image shown in Figure 7.Table 1
Number of circlesN (R) and their radius R for BUS image shown in Figure 7.
Figure7(a)(b)(c)(d)(e)(f)(g)(h)R4567891011N (R)10985726153464337Figure 8
Computation of SC score for the measurements recorded in Table1 on equation (15).


## 4. Deep Learning Features (DLF) Extraction and SVM-Based Classification

In the above, we identified handcrafted features such as shape score, PS score, and EI and orientation to identify the characteristic features (CFs) of the segmented lesion in a BUS image. Here we retrieved high-dimension features from the BUS image to assist the classification. A deep learning method can fix this problem by performing convolution operations on the input graphics multiple times. Therefore, by using deep learning, we extracted the DLF of the image and combined them with the manually extracted CF to distinguish benign and malignant BUS images.Due to the limited dataset, our paper used transfer learning [28] to train the neural network as the DLF extractor. Initially, we compared the classification ability of VGG16 [29], VGG19 [29], ResNet50 [30], and Inception V3 [31] for BUS images. The experiment results show that VGG16 is better than other networks. Then, based on transfer learning, we froze the convolution layer of VGG16, added the global average pooling layer, and changed 7 × 7 × 512 of the original VGG16 output into 1 × 1 × 512. For the full connected layer, the nodes were set to 512, 128, 4, and 2, respectively. At the same time, the Relu activation function was added after each fully connected layer except the last one. The modified VGG16 is shown in Figure 9. For the output, the cross-entropy loss function was used to calculate the loss. After training, for other input images, we could get four values (DLF) from the second fully connected layer of the trained model.Figure 9
Modified VGG16.Once the features have been computed from the BUS images, we used the SVM classifier to classify them as benign or malignant. The SVM was chosen since (a) the availability of labeled large medical image data sets for training is not feasible. SVM could provide better classification accuracy for smaller training sets [32]; (b) SVM theory provides a way to avoid inseparable problem in low-dimensional space through the use of kernel functions [33]. This attribute simplifies the problems in higher-dimensional space providing a more generalized classification.Normally, to solve the problem of linear inseparability in low-dimensional space, kernel functions are used to map the features from low-dimensional space to high-dimensional space, thus realizing the linear classification in higher-dimensional space. In the experiments, the radial basis function (RBF) [34] was used as the kernel function of SVM to classify the CF, the DLF, and the MF, respectively, and the classification results were evaluated by various evaluation indexes.


## 5. Results

For experimental results, we have used 1802 BUS images that include 787 benign and 1015 malignant BUS images. It contains two parts. The first part is provided by Ultrasoundcases.info, which is a professional breast cancer ultrasound website developed by Hitachi Medical Systems in Switzerland and Dr. Taco Geertsma, who works for Gelderse Vallei hospital in the Netherlands. It contains many ultrasound cases, which were collected and labeled by radiologists and ultrasound technicians of the hospital over the years. The other part is supported by the First Affiliated Hospital of Shantou University, Guangdong Province, China.


### 5.1. Evaluation Indexes

The following measures (equations (15)–(19)) were used as a metric to evaluate the performance of the SVM classifier model [35].(15)Sensitivity=TPTP+FN,(16)specificity=TNTN+FP,(17)accuracy=TP+TNTP+TN+FP+FN,(18)F1−score=2TP2TP+FP+FN,(19)precision=TPTP+FP,where true positive (TP): GT malignant and prediction malignant; false positive (FP): GT benign and prediction malignant; false negative (FN): GT malignant and prediction benign; and true negative (TN): GT benign and prediction benign.


### 5.2. Characteristic Feature (CF) Calculation

To illustrate the calculation of CF, we presented 6 BUS images as examples, as shown in Figure10. The calculation results of CF are listed in Table 2. The original value is calculated by using the proposed methods, and the normalized value presents the normalized CF value.Figure 10
BUS images. (a), (b), and (c) are benign. (d), (e), and (f) are malignant.
(a)(b)(c)(d)(e)(f)Table 2
Calculation results of CF in Figure10 (Figures 10(a)–10(c) are benign and Figures 10(d)–10(f) are malignant) (EI—edge indistinctness, PS—posterior shadowing, SC—shape complexity, OV—original value, and NV—normalized value).
ImagesOrientationEIPS scoreSCOVNVOVNVOVNVOVNV(a)0.544027.5300.5722.7511−1.0210(b)0.6700.28327.5290.5721.9830−1.0250.05(c)0.8350.65435.15112.2480.371−1.0270.008(d)0.8780.75117.32902.3210.440−1.0770.7(e)0.989130.1940.7222.4740.693−1.1011(f)0.7720.51223.2150.332.1530.221−1.0340.163It can be seen from Table2 that the characteristic features (CFs) of Figures 10(a) and 10(d) are consistent with the biological properties of benign and malignant BUS images. For example, Figure 10(a) which was diagnosed as benign tumor has larger EI score compared with malignant tumor (Figure 10(d)). However, we can also find that benign and malignant BUS images may have the same properties, for example, the posterior shadowing (PS) shown in Figures 10(b) and 10(f) is relatively low. Therefore, only relying on a single feature to distinguish benign and malignant BUS images will present a higher probability of misjudgment.


### 5.3. Deep Learning Feature (DLF) Extractor

To select the best DLF extractor, based on transfer learning, we used VGG16, VGG19, ResNet50, and Inception V3 to train and test the dataset, which is composed of 955 malignant and 727 benign BUS images. In the experiment, the training set accounts for 80% of the total data and the remaining accounts for the test set. The experimental results are shown in Table3.Table 3
The classification result of using classical deep neural network.
Pretrained modelsAccuracySensitivitySpecificityF1-scoreVGG160.840.860.820.86VGG190.810.810.770.82ResNet500.780.770.770.85InceptionV30.780.830.70.81As we can see from Table3, the accuracy, sensitivity, specificity, and F1-score of VGG16 are 0.84, 0.86, 0.82, and 0.86, respectively, which are higher than those of VGG19, ResNet50, and Inception V3. From the table, it can be inferred that VGG16 has better classification ability; it can learn BUS image features better than other networks. Therefore, we use VGG16 to extract deep learning features (DLFs) to assist the SVM classification.


### 5.4. Classification of Breast Tumors Based on SVM

In this experiment, we used another 120 BUS images, which are totally different from the data used in the above section. Among them, 80 BUS images were used to train the SVM model, including 40 benign BUS images and 40 malignant BUS images. Also, we took another 40 BUS images as the test set. For those 120 BUS images, we extracted their characteristic features (CFs) and deep learning features (DLFs) and then concatenated them serially to form multi-features (MF), as shown in equation (20). Later, the MF was normalized and labeled for supervised learning. We use “1” to indicate malignant and “0” to label benign. After the preparation, we firstly used triple cross-validation on the training samples to get the best classifier and then used the classifier to test the training samples and test samples, respectively.(20)MF=CF,DLF,where,(21)CF=orientation,EI score,PS score,SC score.In Table4, CF and a different number of deep learning features (DLFs) are used as multi-features (MFs) to classify BUS images based on SVM. The result shows that the MF composed of CF and 4 DLF could get better classification results. Therefore, we choose CF and 4 DLF as the final MF. Further, from Table 5, it can be seen that by using SVM, the classification accuracy, precision, sensitivity, specificity, and F1-score of MF are 0.925, 0.905, 0.95, 0.905, and 0.927, respectively, which are higher than those indicators obtained from other classification methods. In Figure 11, the ROC curve and AUC value of SVM classification based on characteristic features (CFs), four deep learning features (DLFs), and multi-features (MFs) are recorded. The results show that the AUC value of MF is 0.970, which is higher than that of CF (0.935) or 4 DLF (0.895). Therefore, the classification model based on MF is better than other classification techniques.Table 4
The comparison of classification using different numbers of DLF and CF (CFs—characteristic features; DLFs—deep learning features).
Classification modelsAccuracySensitivitySpecificityF1-score2 DLF + CF + SVM0.8750.8950.850.8724 DLF + CF + SVM0.9250.9050.950.9276 DLF + CF + SVM0.90.90.90.98 DLF + CF + SVM0.850.8890.80.8410 DLF + CF + SVM0.8750.9410.80.8412 DLF + CF + SVM0.850.8890.80.8416 DLF + CF + SVM0.8750.8950.850.872Table 5
The comparison of classification methods (CFs—characteristic features, DLFs—deep learning features, and MFs—multi-features).
ModelsAccuracyPrecisionSensitivitySpecificityF1-scoreMF + SVM0.9250.9050.950.9050.927CF + SVM0.8750.8950.850.8950.827DLF + SVM0.80.8750.70.8750.778[3]0.901NA0.9350.832NA[7]0.83NANA0.824NAInception V3 [36]0.78NA0.770.78NAVGG19 [36]0.820.700.700.78NA[37]0.8667NA0.92450.7838NAThe bold values indicate that the result of the proposed method is better than that of other classification methods.Figure 11
ROC curve and AUC value by using characteristic features (CFs), deep learning features (DLFs), and multi-features (MFs), respectively.


### 5.5. Triple Cross-Validation

Here, we used triple cross-validation to illustrate the advantages of using the multi-features (MF), that is, for differentc, g values in SVM, the training dataset is randomly divided into three parts: two of them are considered as the training set and the rest is used for verification set. The average accuracy of the three validation sets is considered as the accuracy of the SVM with these c, g values. Here c = 0.5 is the regularization parameter of SVM, and g = 0.25 is the parameter of radial basis function (RBF).The contour plot in Figure12 represents different accuracy values obtained when different c and g were used during triple cross-validation. After the triple cross-validation, the c, g values that represent the highest accuracy among the triple cross-validation were considered as the parameters of the final SVM classifier. The accuracy among the triple cross-validation can reach over 88%, which is close to the final accuracy of 92.5% obtained by using the best c, g. This indeed illustrates that the model has good robustness.Figure 12
Triple cross-validation of multi-features (MFs).


## 5.1. Evaluation Indexes

The following measures (equations (15)–(19)) were used as a metric to evaluate the performance of the SVM classifier model [35].(15)Sensitivity=TPTP+FN,(16)specificity=TNTN+FP,(17)accuracy=TP+TNTP+TN+FP+FN,(18)F1−score=2TP2TP+FP+FN,(19)precision=TPTP+FP,where true positive (TP): GT malignant and prediction malignant; false positive (FP): GT benign and prediction malignant; false negative (FN): GT malignant and prediction benign; and true negative (TN): GT benign and prediction benign.


## 5.2. Characteristic Feature (CF) Calculation

To illustrate the calculation of CF, we presented 6 BUS images as examples, as shown in Figure10. The calculation results of CF are listed in Table 2. The original value is calculated by using the proposed methods, and the normalized value presents the normalized CF value.Figure 10
BUS images. (a), (b), and (c) are benign. (d), (e), and (f) are malignant.
(a)(b)(c)(d)(e)(f)Table 2
Calculation results of CF in Figure10 (Figures 10(a)–10(c) are benign and Figures 10(d)–10(f) are malignant) (EI—edge indistinctness, PS—posterior shadowing, SC—shape complexity, OV—original value, and NV—normalized value).
ImagesOrientationEIPS scoreSCOVNVOVNVOVNVOVNV(a)0.544027.5300.5722.7511−1.0210(b)0.6700.28327.5290.5721.9830−1.0250.05(c)0.8350.65435.15112.2480.371−1.0270.008(d)0.8780.75117.32902.3210.440−1.0770.7(e)0.989130.1940.7222.4740.693−1.1011(f)0.7720.51223.2150.332.1530.221−1.0340.163It can be seen from Table2 that the characteristic features (CFs) of Figures 10(a) and 10(d) are consistent with the biological properties of benign and malignant BUS images. For example, Figure 10(a) which was diagnosed as benign tumor has larger EI score compared with malignant tumor (Figure 10(d)). However, we can also find that benign and malignant BUS images may have the same properties, for example, the posterior shadowing (PS) shown in Figures 10(b) and 10(f) is relatively low. Therefore, only relying on a single feature to distinguish benign and malignant BUS images will present a higher probability of misjudgment.


## 5.3. Deep Learning Feature (DLF) Extractor

To select the best DLF extractor, based on transfer learning, we used VGG16, VGG19, ResNet50, and Inception V3 to train and test the dataset, which is composed of 955 malignant and 727 benign BUS images. In the experiment, the training set accounts for 80% of the total data and the remaining accounts for the test set. The experimental results are shown in Table3.Table 3
The classification result of using classical deep neural network.
Pretrained modelsAccuracySensitivitySpecificityF1-scoreVGG160.840.860.820.86VGG190.810.810.770.82ResNet500.780.770.770.85InceptionV30.780.830.70.81As we can see from Table3, the accuracy, sensitivity, specificity, and F1-score of VGG16 are 0.84, 0.86, 0.82, and 0.86, respectively, which are higher than those of VGG19, ResNet50, and Inception V3. From the table, it can be inferred that VGG16 has better classification ability; it can learn BUS image features better than other networks. Therefore, we use VGG16 to extract deep learning features (DLFs) to assist the SVM classification.


## 5.4. Classification of Breast Tumors Based on SVM

In this experiment, we used another 120 BUS images, which are totally different from the data used in the above section. Among them, 80 BUS images were used to train the SVM model, including 40 benign BUS images and 40 malignant BUS images. Also, we took another 40 BUS images as the test set. For those 120 BUS images, we extracted their characteristic features (CFs) and deep learning features (DLFs) and then concatenated them serially to form multi-features (MF), as shown in equation (20). Later, the MF was normalized and labeled for supervised learning. We use “1” to indicate malignant and “0” to label benign. After the preparation, we firstly used triple cross-validation on the training samples to get the best classifier and then used the classifier to test the training samples and test samples, respectively.(20)MF=CF,DLF,where,(21)CF=orientation,EI score,PS score,SC score.In Table4, CF and a different number of deep learning features (DLFs) are used as multi-features (MFs) to classify BUS images based on SVM. The result shows that the MF composed of CF and 4 DLF could get better classification results. Therefore, we choose CF and 4 DLF as the final MF. Further, from Table 5, it can be seen that by using SVM, the classification accuracy, precision, sensitivity, specificity, and F1-score of MF are 0.925, 0.905, 0.95, 0.905, and 0.927, respectively, which are higher than those indicators obtained from other classification methods. In Figure 11, the ROC curve and AUC value of SVM classification based on characteristic features (CFs), four deep learning features (DLFs), and multi-features (MFs) are recorded. The results show that the AUC value of MF is 0.970, which is higher than that of CF (0.935) or 4 DLF (0.895). Therefore, the classification model based on MF is better than other classification techniques.Table 4
The comparison of classification using different numbers of DLF and CF (CFs—characteristic features; DLFs—deep learning features).
Classification modelsAccuracySensitivitySpecificityF1-score2 DLF + CF + SVM0.8750.8950.850.8724 DLF + CF + SVM0.9250.9050.950.9276 DLF + CF + SVM0.90.90.90.98 DLF + CF + SVM0.850.8890.80.8410 DLF + CF + SVM0.8750.9410.80.8412 DLF + CF + SVM0.850.8890.80.8416 DLF + CF + SVM0.8750.8950.850.872Table 5
The comparison of classification methods (CFs—characteristic features, DLFs—deep learning features, and MFs—multi-features).
ModelsAccuracyPrecisionSensitivitySpecificityF1-scoreMF + SVM0.9250.9050.950.9050.927CF + SVM0.8750.8950.850.8950.827DLF + SVM0.80.8750.70.8750.778[3]0.901NA0.9350.832NA[7]0.83NANA0.824NAInception V3 [36]0.78NA0.770.78NAVGG19 [36]0.820.700.700.78NA[37]0.8667NA0.92450.7838NAThe bold values indicate that the result of the proposed method is better than that of other classification methods.Figure 11
ROC curve and AUC value by using characteristic features (CFs), deep learning features (DLFs), and multi-features (MFs), respectively.


## 5.5. Triple Cross-Validation

Here, we used triple cross-validation to illustrate the advantages of using the multi-features (MF), that is, for differentc, g values in SVM, the training dataset is randomly divided into three parts: two of them are considered as the training set and the rest is used for verification set. The average accuracy of the three validation sets is considered as the accuracy of the SVM with these c, g values. Here c = 0.5 is the regularization parameter of SVM, and g = 0.25 is the parameter of radial basis function (RBF).The contour plot in Figure12 represents different accuracy values obtained when different c and g were used during triple cross-validation. After the triple cross-validation, the c, g values that represent the highest accuracy among the triple cross-validation were considered as the parameters of the final SVM classifier. The accuracy among the triple cross-validation can reach over 88%, which is close to the final accuracy of 92.5% obtained by using the best c, g. This indeed illustrates that the model has good robustness.Figure 12
Triple cross-validation of multi-features (MFs).


## 6. Discussion

Recently, various models relied on deep learning that has been used in BUS image classification. For example, references [36, 38–40] used deep learning methods to extract BUS image features and present classification. Compared with the direct use of traditional deep learning models such as VGG16, VGG19, ResNet50, and Inception V3, they got better results. However, they have the following problems. First of all, because CAD system is mainly used to assist doctors in BI-RADS classification of breast tumors, rather than just relying on CAD system to determine benign and malignant breast tumors, the quantitative methods we proposed can better assist doctors in diagnosis. In addition, according to the clinical experience of doctors, on the basis of segmentation, we quantify the regions with different characteristics of benign and malignant breast tumors, such as posterior shadowing (PS), edge indistinctness (EI), and so on, so as to avoid the influence of irrelevant areas in BUS images on the experimental results and reduce the interference of inherent noise of BUS images.


## 7. Conclusions

In this work, firstly, with the help of the clinical experience of doctors, four characteristic features (CFs) of BUS images were obtained manually, including orientation, edge indistinctness (EI), characteristics of posterior shadowing region (PS), and shape complexity (SC). Based on the experiments, we compared the CF computed from different BUS images, which showed that the CF designed in this paper can characterize the different properties of benign and malignant BUS images. Meanwhile, the experiment showed that using a single feature to distinguish BUS images was prone to false interpretation. Also, this paper introduced deep learning features (DLFs) to improve the accuracy of classification further. In the experiment of DLF extraction, through comparing the classification results of several classical deep neural networks, it was found that the accuracy, sensitivity, specificity, andF1-score of VGG16 are 0.84, 0.86, 0.82, and 0.86, respectively, which were higher than those of other classical neural networks. Therefore, we employed a modified VGG16 as a deep learning feature extractor followed by SVM to classify the fused the CF and DLF. The results showed that the accuracy, sensitivity, specificity, and F1-score of this method are 92.5%, 90.5%, 95%, 90.5%, and 92.7% respectively, which were better than other methods. Finally, through the triple cross-validation of multi-features (MFs), the experiment results further indicated that the proposed method can be used to assist doctors to identify benign and malignant BUS images effectively.



---

*Source: 9980326-2021-05-19.xml*