# Ultrasonic Image Diagnosis of Liver and Spleen Injury Based on a Double-Channel Convolutional Neural Network

Authors: Maorui He; Rui Zhang; Shuni Liu; Yansong Tan; Yang Zeng
Journal: Wireless Communications and Mobile Computing
 (2021)

---

## Abstract

Automatic and accurate diagnosis of liver and spleen injury in ultrasonic images is of great significance for the development of automatic clinical diagnosis. In order to realize more accurate ultrasonic image diagnosis of liver and spleen injury, an algorithm of ultrasonic image classification diagnosis of liver and spleen injury based on double-channel convolutional neural network was proposed. Firstly, the anisotropic diffusion denoising model is used to realize data preprocessing of ultrasonic images of the liver and spleen to improve the image quality of ultrasonic images. Secondly, the external edge of the lesion location was detected to obtain the characteristics of the external edge. Then, the rotation invariant local binary mode feature of the extracted image is taken as the inner texture feature of the image. Finally, the external edge feature and internal texture feature are used as two input channels of the convolutional neural network, respectively, to classify and identify ultrasonic images of liver and spleen injury. The experimental results show that the proposed method diagnoses liver and spleen injury more accurately.


---

## Body

## 1. Introduction

Ultrasound imaging technology is widely used in clinical medicine due to its nonradiation damage and noninvasive features, such as the observation and diagnosis of the liver, gallbladder, spleen, kidney, and other vital organs of the human body. For sensitive groups such as pregnant women and children, the application of ultrasound imaging technology is particularly important [1].Liver and spleen injury is classified into 6 levels, according to the American Association for the Surgery of Trauma (AAST) [2] classification method, which are shown in Table 1.Table 1
The AAST liver trauma grading system.
ClassificationDetail descriptionLevel IHematoma: subcapsular, <10% surface areaLaceration: capsular tear, <1 cm parenchymal depthLevel IIHematoma: subcapsular, 10-50% surface areaHematoma: intraparenchymal, <10 cm diameterLaceration: capsular tear 1-3 cm parenchymal depth, <10 cm lengthLevel IIIHematoma: subcapsular, >50% surface area of ruptured subcapsular or parenchymal hematomaHematoma: intraparenchymal, >10 cmLaceration: capsular tear, >3 cm parenchymal depthVascular injury with active bleeding contained within liver parenchymaLevel IVLaceration: parenchymal disruption involving 25-75% hepatic lobe or involves 1-3 Couinaud segmentsVascular injury with active bleeding breaching the liver parenchyma into the peritoneumLevel VLaceration: parenchymal disruption involving >75% of hepatic lobeVascular: juxtahepatic venous injuries (retrohepatic vena cava/central major hepatic veins)Level VIHepatic avulsion: this vascular injury is rarely seen on imaging due to its high mortality rate; no case could be featured as a resultIt is the key to provide reasonable treatment for liver and spleen injury complicated with many injuries, complicated injuries, and dangerous conditions. At present, texture features commonly used in image recognition have been successfully applied in the real world, including spatial gray scale independent matrix [3], Fourier energy spectrum [4], gray scale difference statistics [5], and laws texture energy measurement [6]. The image characteristics extracted by M-band wavelet transform and fractal characteristics are adopted to represent liver diseases [7]. His study proved that fractal feature vectors could provide better classification performance than other features, and could also distinguish normal and abnormal liver. Convolutional neural network (CNN) has a significant effect in image recognition, image segmentation, and other image processing applications [8]. Hijazi et al. [9] proposed that CNN has the superiority as follows.(1)
CNN-based detection and feature extraction have good robustness for image degradation of image quality due to changes in camera focal length, ambient light intensity, photo pose, image obscuration, image displacement, etc.(2)
The calculation amount of CNN-based feature extraction algorithm is relatively low, because for different input images, the convolution parameters of each layer network are the sameCNN is widely recognized in general image processing, so it is gradually applied in medical image recognition and classification. In literature [10], CNN is used to identify pathological images of breast cancer. Literature [11] applied CNN to the pulmonary nodule identification system. Literature [12] applied CNN to the lymph node recognition system. From these literatures, we can find that CNN has made impressive application in medical diagnosis. In order to identify the damage of liver and spleen ultrasound images, CNN will be used as an identification network in this paper. Before using CNN, we analyzed CNN model, and its training methods have the disadvantages as follows.(1)
Images were collected from different medical institutions for algorithm training, so the image scale in the data set changed too much(2)
The amount of training image data is not enough, so the ideal CNN network cannot be obtained by independent trainingIt is very necessary to improve the image quality and carry out transfer learning training to the existing convolutional neural network; thus, the network can extract the features in medical images.In order to achieve more accurate ultrasound image diagnosis of liver and spleen injury, this paper proposes a classification and diagnosis algorithm for ultrasound images of liver and spleen injury based on a double-channel convolutional neural network. An algorithm based on dual channel convolution neural network is designed to recognize and diagnose ultrasound images of liver and spleen injury. To increase the signal-to-noise ratio of ultrasound image, the anisotropic diffusion denoising model is used to preprocess the ultrasound image of the liver and spleen. In order to obtain the outer edge feature, this paper detects the outer edge of the lesion location. Then, the rotation invariant local binary mode feature of the extracted image is taken as the internal texture feature of the image. In the process of neural network recognition, the recognition ability of neural network is enhanced by adding key features. Because the clinicians of ultrasound image usually take the lost external edge features and internal texture features as the important basis for diagnosis, so these two points are used as the two input channels of the network in this paper. In the experimental part, the clinical medical ultrasound image is used as a data set and compared with other recognition algorithms to further verify the effectiveness and accuracy of this algorithm.


## 2. The Basic Theory

### 2.1. Convolutional Neural Network

A typical convolutional neural network consists of four layers: input layer, convolution layer, downsampling layer, and output layer. It is a kind of network structure with the characteristics of self-supervised learning. It optimizes the network structure through reverse conduction and then updates the optimal parameters of each layer of the network. In the convolution layer processing, the mapping relationship of multiple characteristics can be obtained, and each feature mapping corresponds to one characteristic extraction. Each neuron in the neural network is connected with the upper and lower layers. In the process of convolution calculation, each neuron shares a set of weights to complete feature extraction. The weights of different feature graphs are different, and each feature graph has its own weights. In the training phase, the method of updating weights by reverse conduction makes the CNN recognition results more accurate. The convolution layer calculation equation can be expressed as follows:(1)xjl=f∑i∈Mjxjl−1∗kijl+bjl,where bj is the offset, and each input map contains an offset. Mj is a collection of input maps. kij is the convolution kernel for convolution processing.The image resolution is reduced and the amount of characteristic maps remains the same by pooling the feature map on the downsampling layer. After convolution operation of convolution layer, the dimension of image features is reduced. At the same time, convolution operation makes the robustness of image displacement, scale, and distortion further improved. The convolution layer is followed by the downsampling layer, and the downsampling formula can be expressed as follows:(2)xjl=fβjldxjl−1+bjl,where the downsampling function is represented by d. bj is the offset and βj is the weight coefficient of downsampling.


### 2.2. GoogLeNet Neural Networks

Among the current popular CNN network, GoogLeNet was confirmed to perform well in image classification [13]. GoogLeNet was the classification champion of ILSVRC in 2014, and its top 5 error rate was 6.7%. In the GoogLeNet structure, the author designs a 22-layer network and proposes a new structure, namely, perception layer. The perception layer is a combination of filters, which is composed of internal network filters and convolution filters with different sizes each other. The internal network filter is linked to the 3 by 3 pixel or 5 by 5 pixel filter. Then, the feedback is sent to the pool and connection layer. The size of each filter layer is different, so the characteristics of input layer with different resolution are processed. Despite its depth, GoogLeNet has 12 times fewer parameters than AlexNet; thus, the training speed is faster and the efficiency is better.Despite GoogLeNet has an impressive recognition effect on natural images, it cannot be directly applied to liver and spleen ultrasonic images. Due to the large difference between the image content and the original training samples, the network could not converge and could not achieve the ideal result of classification and recognition. Thus, the preprocessed liver and spleen ultrasound image samples are input into the pretrained network, and then, the parameters are fine tuned by transfer learning method. The fine tuning makes the network more sensitive to the content of liver and spleen images, and then carries out feature extraction and classification recognition for liver and spleen ultrasound images.


## 2.1. Convolutional Neural Network

A typical convolutional neural network consists of four layers: input layer, convolution layer, downsampling layer, and output layer. It is a kind of network structure with the characteristics of self-supervised learning. It optimizes the network structure through reverse conduction and then updates the optimal parameters of each layer of the network. In the convolution layer processing, the mapping relationship of multiple characteristics can be obtained, and each feature mapping corresponds to one characteristic extraction. Each neuron in the neural network is connected with the upper and lower layers. In the process of convolution calculation, each neuron shares a set of weights to complete feature extraction. The weights of different feature graphs are different, and each feature graph has its own weights. In the training phase, the method of updating weights by reverse conduction makes the CNN recognition results more accurate. The convolution layer calculation equation can be expressed as follows:(1)xjl=f∑i∈Mjxjl−1∗kijl+bjl,where bj is the offset, and each input map contains an offset. Mj is a collection of input maps. kij is the convolution kernel for convolution processing.The image resolution is reduced and the amount of characteristic maps remains the same by pooling the feature map on the downsampling layer. After convolution operation of convolution layer, the dimension of image features is reduced. At the same time, convolution operation makes the robustness of image displacement, scale, and distortion further improved. The convolution layer is followed by the downsampling layer, and the downsampling formula can be expressed as follows:(2)xjl=fβjldxjl−1+bjl,where the downsampling function is represented by d. bj is the offset and βj is the weight coefficient of downsampling.


## 2.2. GoogLeNet Neural Networks

Among the current popular CNN network, GoogLeNet was confirmed to perform well in image classification [13]. GoogLeNet was the classification champion of ILSVRC in 2014, and its top 5 error rate was 6.7%. In the GoogLeNet structure, the author designs a 22-layer network and proposes a new structure, namely, perception layer. The perception layer is a combination of filters, which is composed of internal network filters and convolution filters with different sizes each other. The internal network filter is linked to the 3 by 3 pixel or 5 by 5 pixel filter. Then, the feedback is sent to the pool and connection layer. The size of each filter layer is different, so the characteristics of input layer with different resolution are processed. Despite its depth, GoogLeNet has 12 times fewer parameters than AlexNet; thus, the training speed is faster and the efficiency is better.Despite GoogLeNet has an impressive recognition effect on natural images, it cannot be directly applied to liver and spleen ultrasonic images. Due to the large difference between the image content and the original training samples, the network could not converge and could not achieve the ideal result of classification and recognition. Thus, the preprocessed liver and spleen ultrasound image samples are input into the pretrained network, and then, the parameters are fine tuned by transfer learning method. The fine tuning makes the network more sensitive to the content of liver and spleen images, and then carries out feature extraction and classification recognition for liver and spleen ultrasound images.


## 3. Algorithm Implementation

### 3.1. Image Preprocessing

Liver and spleen ultrasound images are obtained by using ultrasonic signals reflected when ultrasound encounters human tissues. Limited by the imaging mechanism of medical ultrasound images, the noise interference of liver and spleen ultrasound images is serious, mainly manifested as additive thermal noise and multiplicative speckle noise. Thermal noise is caused by the heating of imaging equipment and can be avoided by cooling imaging equipment and other physical means. Speckle noise is the light and dark granular speckle caused by the long interference and the canceling interference generated by the reflected ultrasound, which cannot be eliminated by adjusting the physical equipment. Therefore, the ultrasonic image denoising of the liver and spleen needs to be realized by preprocessing.By combining the diffusion equation and image features [14], the anisotropic diffusion method can retain or even enhance the image edge information while smoothing the image. The noise distribution and strength of liver and spleen ultrasonic images are different in different tissue parts of the human body, so the denoising model should implement isotropic diffusion denoising in homogeneous regions and anisotropic diffusion denoising in heterogeneous regions, so as to avoid reducing the quality of ultrasonic images. Therefore, the EEAD [15] model based on edge enhancement is adopted in this paper to denoise the image. In this model, the diffusion velocity is determined by the image gradient. The filtering diffusion direction is decomposed into two parts that are the normal direction and the tangent direction. The diffusion equation is as follows:(3)∂I∂t=divc∇I∇I=f1Inn+f2It,f1=11+avn2+bvnn2,f2=11+avn2+bvnn2,v=Gt⋅I,where I is the original image, t is a time iteration parameter, c∇I is a margin stop function also known as flow diffusion coefficient, and ∇ is a gradient operator. The subscripts t and n are the first-order partial derivatives of the image in the direction of unit tangent and normal vector. The subscripts tt and nn stand for second-order partial derivatives of the image in the direction of unit tangent and normal vector. f1 is the diffusion coefficient in the normal direction, f2 is the diffusion coefficient in the tangent direction. a is the retention coefficient of edge and local details when anisotropic diffusion is controlled for denoising, b is the retention coefficient of ultrasonic echo bright bar when anisotropic diffusion is controlled for denoising, Gt is Gaussian smoothing function, and v is the smoothed image.In order to maintain useful information in ultrasonic images, mean square error (MSE) is used as an objective quantitative standard for image quality, and set the parametersf1<f2, f1⟶0, and f2⟶0.


### 3.2. Extract Texture Features

Compared with other feature extraction methods, uniform local binary pattern (ULBP) has the characteristics of simple calculation and good robustness [16]. ULBP can be used as shallow image features to accurately describe texture features in ultrasonic images. The local binary pattern encoding of pixels in image xc,yc is defined as follows:(4)BPRxc,yc=∑p=0PSnp−nc×2p,where S· is defined as follows:(5)Sx=1,x≥TLBP,0,x≺TLBP,where nc is the pixel value of the neighborhood center point xc,yc. On the circle of radius R, nP is the value of the pth pixel on the circle. In order to enhance the robustness of LBP in describing the smooth region of the image, TLBP is set as a small and positive value.The local binary mode with rotation invariance can be calculated as follows. First, the LBP code of any pixel is calculated, and then, the LBP translation code of the minimum value is obtained by rotating it to the right. The minimum LBP translation code of each pixel in the image is calculated. Finally, the histogram of local binary mode translation value is used as the shallow texture characteristic value of the input image.For the ultrasound images of the liver and spleen to be processed, the images are input into GoogLeNet as input information, and the output is the depth features of the images. For the shallow texture features of the image, ULBP is used as the texture feature extraction method. In order to integrate the two characteristics of deep layer and shallow layer, they are normalized, respectively. After normalization processing, the high and low level image features are as follows:(6)F^high=Fhigh−FhighminFhighmax−Fhighmin,(7)F^low=Flow−FlowminFlowmax−Flowmin,(8)Flow=Fhigh,Flow,where F^high is the depth characteristics Fhigh after a normalized value. F^highmin and F^highmax are the minimum and maximum values of the prenormalized depth eigenvectors, respectively. F^low is the normalized value. Flow represents the unnormalized value. Flowmin expressed the minimum value of the unnormalized characteristic vector. Fhighmin is maximum value of the unnormalized characteristic vector, while F is the fused characteristic, which is used for ultrasonic image recognition and classification.


### 3.3. Segment Edge Information

Among the numerous image edge detection methods for image edge detection, the common edge detection operators are Sobel, Log, Prewitt, and Roberts. The implementation of these algorithms is relatively simple and the detection is fast, but they are vulnerable to noise interference. If it is applied to remote sensing image edge detection, there will be many interference edges, edge discontinuity, or loss of road details.Canny edge detection method based on optimization idea can make up for the shortcomings of other gradient operators, and is considered to be the most successful and widely used gray edge detection method [17]. Canny method is mainly implemented in four steps.(1)
Smooth image denoising. The Gaussian function g (x,y) is used to convolute the image dx,y to get the smooth image sx,y:(9)Gx,y=12πσexp−x2+y22σ2,(10)sx,y=dx,y×Gx,y,where σ is the smoothing scale parameter.(2)
The gradient amplitude and direction are calculated. The appropriate gradient operator is selected to calculate the gradient size and direction of each pixel after noise reduction(3)
Nonmaximum suppression (NMS). In order to locate the edge points accurately, the gradient value of each pixel is suppressed by nonmaximum method. In the neighborhood of the current pixel, the gradient amplitude of the point is compared. If the gradient amplitude is greater than the gray value of two adjacent pixels along the gradient direction, then the point is judged to be a possible edge point. Otherwise, the point is judged as nonedge point(4)
High and low threshold detection and edge connection. Through the above steps, the edge is only rough. Then, the false edge points are removed by high and low threshold detection. The points smaller than the low threshold are excluded, and the points larger than the high threshold are determined as edge points. If it is between the two, it will be marked as weak edge point, and then judge whether the weak edge point is connected with the edge point. If the weak edge point is connected with the edge point, the point is recorded as the edge pointThe amplitude and direction of the gradient are calculated by using Canny algorithm with the traditional template of size (2×2). However, it is greatly affected by the noise and cannot accurately detect the edge. Therefore, on this basis, this paper adopts Sobel edge detection operator to calculate the first partial derivative from the four directions of horizontal, vertical, 45°, and 135°, respectively. Good results are achieved in the experiment. The template is shown in Figure 1.Figure 1
Gradient direction template diagram of four directions.
(a)
Directions of horizontal(b)
Directions of vertical(c)
Directions of 45°(d)
Directions of 135°


### 3.4. Classification of Ultrasound Images of Liver and Spleen Injury with Double-Channel CNN

In the structure of convolutional neural network, the first layer is the input layer, namely, the input image. The feature extracted from the first convolution layer behind the input layer is the feature graph of the first layer. Since the first layer feature map is the basis of subsequent feature map and feature used for classification, the first convolution layer is very important to the degree of image understanding. In the convolution layer, the size of the convolution kernel determines the size of the granularity of the network recognition image. If the size of the convolution kernel is too large, some details of the edge contour of the feature will be lost; if the size of the convolution kernel is too small, the ability of the convolution operation to transform the feature will be reduced, and the feature cannot be extracted effectively. Therefore, the size of the convolution kernel is very important to the effect of network feature extraction.The deep features extracted by convolutional neural network are combined by the shallow features, so the loss of the first layer features will lead to the failure of subsequent networks to obtain effective deep features. In traditional convolutional neural networks, only one input channel is usually designed. Moreover, the kernel size of the first convolution layer is set to a fixed value, so the initial image recognition granularity of this neural network is fixed. However, for different kinds of images to be processed, the original features generally do not exist in a fixed size. Therefore, the design of a single convolutional kernel will lead to the loss of features of other scales of the original image; that is, the single-channel convolutional neural network cannot adequately extract the eigenvalues of the original image.Aiming at the problem of insufficient feature extraction in traditional single-channel convolutional neural network, a dual-channel convolutional neural network for ultrasonic image diagnosis is proposed in this paper. The network consists of two relatively independent network structures. Each network is composed of an input layer, a convolution layer, a pooling layer, and a full connection layer. In the input layer, external edge features and internal texture features are taken as input information of the dual channel convolutional neural network, respectively. In the first layer of the convolution layer, the size of the convolution kernel is set according to the characteristics, respectively, so the two networks usually use the convolution check of different sizes to extract the two input layers. Compared with the maximum pooling method, the dynamicK-max pooling method can not only retain a number of important features of the image but also improve the training efficiency of the network structure. Therefore, the K-max pooling method is chosen in this paper. After the pooling layer, the two channels are connected to one full connection layer for a full connection mapping, and then, the two full connection layers are combined together through one full connection layer, followed by the Softmax classifier to classify the extracted characteristics.In clinical diagnosis, the external edge information and internal texture information of the liver and spleen are often used as the important basis for doctors’ diagnosis. Therefore, the external edge feature and internal texture feature are, respectively, taken as the two input channels of the convolutional neural network in this paper. Then, combined with the dynamicK-max pooling method, this paper proposed an ultrasonic image classification algorithm for liver and spleen injury with the double-channel CNN.


## 3.1. Image Preprocessing

Liver and spleen ultrasound images are obtained by using ultrasonic signals reflected when ultrasound encounters human tissues. Limited by the imaging mechanism of medical ultrasound images, the noise interference of liver and spleen ultrasound images is serious, mainly manifested as additive thermal noise and multiplicative speckle noise. Thermal noise is caused by the heating of imaging equipment and can be avoided by cooling imaging equipment and other physical means. Speckle noise is the light and dark granular speckle caused by the long interference and the canceling interference generated by the reflected ultrasound, which cannot be eliminated by adjusting the physical equipment. Therefore, the ultrasonic image denoising of the liver and spleen needs to be realized by preprocessing.By combining the diffusion equation and image features [14], the anisotropic diffusion method can retain or even enhance the image edge information while smoothing the image. The noise distribution and strength of liver and spleen ultrasonic images are different in different tissue parts of the human body, so the denoising model should implement isotropic diffusion denoising in homogeneous regions and anisotropic diffusion denoising in heterogeneous regions, so as to avoid reducing the quality of ultrasonic images. Therefore, the EEAD [15] model based on edge enhancement is adopted in this paper to denoise the image. In this model, the diffusion velocity is determined by the image gradient. The filtering diffusion direction is decomposed into two parts that are the normal direction and the tangent direction. The diffusion equation is as follows:(3)∂I∂t=divc∇I∇I=f1Inn+f2It,f1=11+avn2+bvnn2,f2=11+avn2+bvnn2,v=Gt⋅I,where I is the original image, t is a time iteration parameter, c∇I is a margin stop function also known as flow diffusion coefficient, and ∇ is a gradient operator. The subscripts t and n are the first-order partial derivatives of the image in the direction of unit tangent and normal vector. The subscripts tt and nn stand for second-order partial derivatives of the image in the direction of unit tangent and normal vector. f1 is the diffusion coefficient in the normal direction, f2 is the diffusion coefficient in the tangent direction. a is the retention coefficient of edge and local details when anisotropic diffusion is controlled for denoising, b is the retention coefficient of ultrasonic echo bright bar when anisotropic diffusion is controlled for denoising, Gt is Gaussian smoothing function, and v is the smoothed image.In order to maintain useful information in ultrasonic images, mean square error (MSE) is used as an objective quantitative standard for image quality, and set the parametersf1<f2, f1⟶0, and f2⟶0.


## 3.2. Extract Texture Features

Compared with other feature extraction methods, uniform local binary pattern (ULBP) has the characteristics of simple calculation and good robustness [16]. ULBP can be used as shallow image features to accurately describe texture features in ultrasonic images. The local binary pattern encoding of pixels in image xc,yc is defined as follows:(4)BPRxc,yc=∑p=0PSnp−nc×2p,where S· is defined as follows:(5)Sx=1,x≥TLBP,0,x≺TLBP,where nc is the pixel value of the neighborhood center point xc,yc. On the circle of radius R, nP is the value of the pth pixel on the circle. In order to enhance the robustness of LBP in describing the smooth region of the image, TLBP is set as a small and positive value.The local binary mode with rotation invariance can be calculated as follows. First, the LBP code of any pixel is calculated, and then, the LBP translation code of the minimum value is obtained by rotating it to the right. The minimum LBP translation code of each pixel in the image is calculated. Finally, the histogram of local binary mode translation value is used as the shallow texture characteristic value of the input image.For the ultrasound images of the liver and spleen to be processed, the images are input into GoogLeNet as input information, and the output is the depth features of the images. For the shallow texture features of the image, ULBP is used as the texture feature extraction method. In order to integrate the two characteristics of deep layer and shallow layer, they are normalized, respectively. After normalization processing, the high and low level image features are as follows:(6)F^high=Fhigh−FhighminFhighmax−Fhighmin,(7)F^low=Flow−FlowminFlowmax−Flowmin,(8)Flow=Fhigh,Flow,where F^high is the depth characteristics Fhigh after a normalized value. F^highmin and F^highmax are the minimum and maximum values of the prenormalized depth eigenvectors, respectively. F^low is the normalized value. Flow represents the unnormalized value. Flowmin expressed the minimum value of the unnormalized characteristic vector. Fhighmin is maximum value of the unnormalized characteristic vector, while F is the fused characteristic, which is used for ultrasonic image recognition and classification.


## 3.3. Segment Edge Information

Among the numerous image edge detection methods for image edge detection, the common edge detection operators are Sobel, Log, Prewitt, and Roberts. The implementation of these algorithms is relatively simple and the detection is fast, but they are vulnerable to noise interference. If it is applied to remote sensing image edge detection, there will be many interference edges, edge discontinuity, or loss of road details.Canny edge detection method based on optimization idea can make up for the shortcomings of other gradient operators, and is considered to be the most successful and widely used gray edge detection method [17]. Canny method is mainly implemented in four steps.(1)
Smooth image denoising. The Gaussian function g (x,y) is used to convolute the image dx,y to get the smooth image sx,y:(9)Gx,y=12πσexp−x2+y22σ2,(10)sx,y=dx,y×Gx,y,where σ is the smoothing scale parameter.(2)
The gradient amplitude and direction are calculated. The appropriate gradient operator is selected to calculate the gradient size and direction of each pixel after noise reduction(3)
Nonmaximum suppression (NMS). In order to locate the edge points accurately, the gradient value of each pixel is suppressed by nonmaximum method. In the neighborhood of the current pixel, the gradient amplitude of the point is compared. If the gradient amplitude is greater than the gray value of two adjacent pixels along the gradient direction, then the point is judged to be a possible edge point. Otherwise, the point is judged as nonedge point(4)
High and low threshold detection and edge connection. Through the above steps, the edge is only rough. Then, the false edge points are removed by high and low threshold detection. The points smaller than the low threshold are excluded, and the points larger than the high threshold are determined as edge points. If it is between the two, it will be marked as weak edge point, and then judge whether the weak edge point is connected with the edge point. If the weak edge point is connected with the edge point, the point is recorded as the edge pointThe amplitude and direction of the gradient are calculated by using Canny algorithm with the traditional template of size (2×2). However, it is greatly affected by the noise and cannot accurately detect the edge. Therefore, on this basis, this paper adopts Sobel edge detection operator to calculate the first partial derivative from the four directions of horizontal, vertical, 45°, and 135°, respectively. Good results are achieved in the experiment. The template is shown in Figure 1.Figure 1
Gradient direction template diagram of four directions.
(a)
Directions of horizontal(b)
Directions of vertical(c)
Directions of 45°(d)
Directions of 135°


## 3.4. Classification of Ultrasound Images of Liver and Spleen Injury with Double-Channel CNN

In the structure of convolutional neural network, the first layer is the input layer, namely, the input image. The feature extracted from the first convolution layer behind the input layer is the feature graph of the first layer. Since the first layer feature map is the basis of subsequent feature map and feature used for classification, the first convolution layer is very important to the degree of image understanding. In the convolution layer, the size of the convolution kernel determines the size of the granularity of the network recognition image. If the size of the convolution kernel is too large, some details of the edge contour of the feature will be lost; if the size of the convolution kernel is too small, the ability of the convolution operation to transform the feature will be reduced, and the feature cannot be extracted effectively. Therefore, the size of the convolution kernel is very important to the effect of network feature extraction.The deep features extracted by convolutional neural network are combined by the shallow features, so the loss of the first layer features will lead to the failure of subsequent networks to obtain effective deep features. In traditional convolutional neural networks, only one input channel is usually designed. Moreover, the kernel size of the first convolution layer is set to a fixed value, so the initial image recognition granularity of this neural network is fixed. However, for different kinds of images to be processed, the original features generally do not exist in a fixed size. Therefore, the design of a single convolutional kernel will lead to the loss of features of other scales of the original image; that is, the single-channel convolutional neural network cannot adequately extract the eigenvalues of the original image.Aiming at the problem of insufficient feature extraction in traditional single-channel convolutional neural network, a dual-channel convolutional neural network for ultrasonic image diagnosis is proposed in this paper. The network consists of two relatively independent network structures. Each network is composed of an input layer, a convolution layer, a pooling layer, and a full connection layer. In the input layer, external edge features and internal texture features are taken as input information of the dual channel convolutional neural network, respectively. In the first layer of the convolution layer, the size of the convolution kernel is set according to the characteristics, respectively, so the two networks usually use the convolution check of different sizes to extract the two input layers. Compared with the maximum pooling method, the dynamicK-max pooling method can not only retain a number of important features of the image but also improve the training efficiency of the network structure. Therefore, the K-max pooling method is chosen in this paper. After the pooling layer, the two channels are connected to one full connection layer for a full connection mapping, and then, the two full connection layers are combined together through one full connection layer, followed by the Softmax classifier to classify the extracted characteristics.In clinical diagnosis, the external edge information and internal texture information of the liver and spleen are often used as the important basis for doctors’ diagnosis. Therefore, the external edge feature and internal texture feature are, respectively, taken as the two input channels of the convolutional neural network in this paper. Then, combined with the dynamicK-max pooling method, this paper proposed an ultrasonic image classification algorithm for liver and spleen injury with the double-channel CNN.


## 4. Experimental Results and Analysis

All the experiments in this paper are implemented under the framework of Python 3 and tensor flow. The CPU of the computer is Intel Core i7 processor. GPU is used to improve the computing speed. The GPU model is GTX965M, and the GPU memory is 8G.


### 4.1. Experimental Data

In this study, the data set we used was obtained by SAMSUNGRS80A color Doppler ultrasonography, with a total of 1200 images. These instrumental images clearly show the morphology, internal structure, and surrounding tissue of liver and spleen injury. Figure2 is an example of liver ultrasound image, and Figure 3 is an example of spleen ultrasound image.Figure 2
Ultrasound image of the liver.Figure 3
Ultrasound image of the spleen.In the process of algorithm training, the ultrasonic image data is first divided into two sample sets of training and testing, and then, the neural network training is used to reach the acceptable range of error. In this experiment, MATLAB neural network toolbox was used to classify the level of ultrasonic image injury of the liver and spleen. The edge information and texture information are input, and the six levels of damage are output. 1200 images were selected for the experiment, including 600 images as training data and 600 images as test data.According to AAST diagnostic criteria shown in Figure4, the liver and spleen morphological injuries were classified into levels I to VI. However, level VI injury is rarely seen on imaging due to its high mortality rate, and no case could be featured as a result. Therefore, the experimental data in this paper only includes 5 levels.Figure 4
The schematic diagram of liver injury.


### 4.2. Comparative Experiment

The feasibility of the two-channel convolution recognition algorithm proposed in this paper is verified at first. For the same experimental data set, a single channel network and a multichannel network were used for comparative experiments. The feasibility of the proposed algorithm is verified by comparing the networks designed with different parameters. At the same time, according to the identification results of various parameters, the most appropriate network parameters are selected as the recommended parameters of the algorithm in this paper. In order to further verify the performance comparison between the algorithm in this paper and other algorithms, a comparative experiment of different algorithms will be carried out. The most appropriate parameters are selected as the premise, and then compared with other algorithms, so as to analyze the advantages of the algorithm in this paper.


#### 4.2.1. Validation of Proposed Double-Channel CNN Algorithm

In the convolutional layer of convolutional neural network, the effect of feature extraction of convolutional check with different sizes is different. In general, different sizes of convolutional kernels are set according to their characteristics when designing convolutional neural networks. After setting the appropriate size of the convolution kernel, the convolution layer can extract more sufficient and effective eigenvalues. Only when the feature extraction is sufficiently effective, can it provide effective guarantee for the subsequent output layer. In order to analyze the algorithm recognition rate of the proposed algorithm under different convolutional kernel sizes, the convolutional kernel size of the single-channel convolutional neural network was first set, and the feature extraction effect was observed. In the single-channel convolutional neural network A,3∗3, 5∗5, and 7∗7 are, respectively, set as the convolutional kernel in order to extract the texture information of the damaged site. In the single-channel convolutional neural network B, 3∗3, 5∗5, and 7∗7 are, respectively, set as convolutional cores in order to extract the edge information of the damaged site. The recognition results of single-channel convolutional neural network are shown in Table 2. It can be seen from Table 2 that in single-channel convolutional neural network A, the recognition result of 5∗5 convolutional kernel is higher; that is, its extraction of texture features is more effective. In single-channel convolutional neural network B, the recognition result of 7∗7 convolutional kernel is higher; that is, its extraction of edge information is better.Table 2
The recognition results of the double-channel and single-channel CNN.
Model structureLevel ILevel IILevel IIILevel IVLevel VSingle-channel CNN A (3∗3)81.2581.7482.3680.7480.53Single-channel CNN A (5∗5)83.8483.1783.4884.2585.3Single-channel CNN A (7∗7)82.3681.7682.8282.8580.94Single-channel CNN B (3∗3)80.1380.6281.2479.6279.41Single-channel CNN B (5∗5)81.3580.7581.8181.8479.93Single-channel CNN B (7∗7)82.6281.9582.2683.0384.08Double-channel CNN (5∗5 and 7∗7)94.9693.3394.6394.8292.40When setting the parameters of the two-channel convolutional neural network proposed in this paper,5∗5 convolutional kernel is selected as the convolutional kernel of the texture information input channel, and 7∗7 convolutional kernel is selected as the convolutional kernel of the edge information input channel. The results of the two-channel convolutional neural network in this paper are shown in the last column of Table 2. By comparing with single-channel convolutional neural network, it can be seen that the two-channel convolutional neural network algorithm proposed in this paper has a higher recognition rate.


#### 4.2.2. Comparison Experiment

In order to compare the recognition performance of the algorithm presented in this paper, the damage recognition algorithms with excellent performance in recent years are selected for comparative experiments under the same experimental data. According to the experimental results above, the convolutional kernel size of the two-channel convolutional neural network designed in this paper is set as5∗5 and 7∗7. The damage recognition results of different algorithms are shown in Table 3.Table 3
The recognition results of different algorithms.
AlgorithmsLevel ILevel IILevel IIILevel IVLevel VLiterature [18]86.3382.5283.5284.2587.36Literature [19]88.4283.4186.2887.1382.74Literature [20]92.4191.9293.4991.3190.37Proposed method94.9693.3394.6394.8292.40Literature [18] proposed a method using multifractal dimension and back propagation neural network. Literature [19] used the deep learning framework to distinguish the unique types of acquired lesions and nodules of the breast through ultrasound imaging. Literature [20] proposed a stacked deep polynomial network (S-DPN) algorithm to improve the representation performance of the original DPN, and the S-DPN algorithm was applied to the texture feature learning of ultrasonic tumor classification in small data sets. Compared with other methods, the recognition rate of this algorithm is higher. This is because the proposed algorithm uses anisotropic diffusion denoising model to preprocess the data of ultrasonic images of the liver and spleen, which improves the image quality. Then, the external edge feature and the internal texture feature are taken as the double-input channels of the convolutional neural network, respectively, which makes the extracted features more abundant and conducive to recognition, so the recognition rate is higher. Based on the above results, it can be seen that the double-channel convolutional neural network can learn more features by using the two input channels of texture information and edge information of damaged images. Compared with other damage identification algorithms, double-channel CNN has higher accuracy in damage identification.


## 4.1. Experimental Data

In this study, the data set we used was obtained by SAMSUNGRS80A color Doppler ultrasonography, with a total of 1200 images. These instrumental images clearly show the morphology, internal structure, and surrounding tissue of liver and spleen injury. Figure2 is an example of liver ultrasound image, and Figure 3 is an example of spleen ultrasound image.Figure 2
Ultrasound image of the liver.Figure 3
Ultrasound image of the spleen.In the process of algorithm training, the ultrasonic image data is first divided into two sample sets of training and testing, and then, the neural network training is used to reach the acceptable range of error. In this experiment, MATLAB neural network toolbox was used to classify the level of ultrasonic image injury of the liver and spleen. The edge information and texture information are input, and the six levels of damage are output. 1200 images were selected for the experiment, including 600 images as training data and 600 images as test data.According to AAST diagnostic criteria shown in Figure4, the liver and spleen morphological injuries were classified into levels I to VI. However, level VI injury is rarely seen on imaging due to its high mortality rate, and no case could be featured as a result. Therefore, the experimental data in this paper only includes 5 levels.Figure 4
The schematic diagram of liver injury.


## 4.2. Comparative Experiment

The feasibility of the two-channel convolution recognition algorithm proposed in this paper is verified at first. For the same experimental data set, a single channel network and a multichannel network were used for comparative experiments. The feasibility of the proposed algorithm is verified by comparing the networks designed with different parameters. At the same time, according to the identification results of various parameters, the most appropriate network parameters are selected as the recommended parameters of the algorithm in this paper. In order to further verify the performance comparison between the algorithm in this paper and other algorithms, a comparative experiment of different algorithms will be carried out. The most appropriate parameters are selected as the premise, and then compared with other algorithms, so as to analyze the advantages of the algorithm in this paper.


### 4.2.1. Validation of Proposed Double-Channel CNN Algorithm

In the convolutional layer of convolutional neural network, the effect of feature extraction of convolutional check with different sizes is different. In general, different sizes of convolutional kernels are set according to their characteristics when designing convolutional neural networks. After setting the appropriate size of the convolution kernel, the convolution layer can extract more sufficient and effective eigenvalues. Only when the feature extraction is sufficiently effective, can it provide effective guarantee for the subsequent output layer. In order to analyze the algorithm recognition rate of the proposed algorithm under different convolutional kernel sizes, the convolutional kernel size of the single-channel convolutional neural network was first set, and the feature extraction effect was observed. In the single-channel convolutional neural network A,3∗3, 5∗5, and 7∗7 are, respectively, set as the convolutional kernel in order to extract the texture information of the damaged site. In the single-channel convolutional neural network B, 3∗3, 5∗5, and 7∗7 are, respectively, set as convolutional cores in order to extract the edge information of the damaged site. The recognition results of single-channel convolutional neural network are shown in Table 2. It can be seen from Table 2 that in single-channel convolutional neural network A, the recognition result of 5∗5 convolutional kernel is higher; that is, its extraction of texture features is more effective. In single-channel convolutional neural network B, the recognition result of 7∗7 convolutional kernel is higher; that is, its extraction of edge information is better.Table 2
The recognition results of the double-channel and single-channel CNN.
Model structureLevel ILevel IILevel IIILevel IVLevel VSingle-channel CNN A (3∗3)81.2581.7482.3680.7480.53Single-channel CNN A (5∗5)83.8483.1783.4884.2585.3Single-channel CNN A (7∗7)82.3681.7682.8282.8580.94Single-channel CNN B (3∗3)80.1380.6281.2479.6279.41Single-channel CNN B (5∗5)81.3580.7581.8181.8479.93Single-channel CNN B (7∗7)82.6281.9582.2683.0384.08Double-channel CNN (5∗5 and 7∗7)94.9693.3394.6394.8292.40When setting the parameters of the two-channel convolutional neural network proposed in this paper,5∗5 convolutional kernel is selected as the convolutional kernel of the texture information input channel, and 7∗7 convolutional kernel is selected as the convolutional kernel of the edge information input channel. The results of the two-channel convolutional neural network in this paper are shown in the last column of Table 2. By comparing with single-channel convolutional neural network, it can be seen that the two-channel convolutional neural network algorithm proposed in this paper has a higher recognition rate.


### 4.2.2. Comparison Experiment

In order to compare the recognition performance of the algorithm presented in this paper, the damage recognition algorithms with excellent performance in recent years are selected for comparative experiments under the same experimental data. According to the experimental results above, the convolutional kernel size of the two-channel convolutional neural network designed in this paper is set as5∗5 and 7∗7. The damage recognition results of different algorithms are shown in Table 3.Table 3
The recognition results of different algorithms.
AlgorithmsLevel ILevel IILevel IIILevel IVLevel VLiterature [18]86.3382.5283.5284.2587.36Literature [19]88.4283.4186.2887.1382.74Literature [20]92.4191.9293.4991.3190.37Proposed method94.9693.3394.6394.8292.40Literature [18] proposed a method using multifractal dimension and back propagation neural network. Literature [19] used the deep learning framework to distinguish the unique types of acquired lesions and nodules of the breast through ultrasound imaging. Literature [20] proposed a stacked deep polynomial network (S-DPN) algorithm to improve the representation performance of the original DPN, and the S-DPN algorithm was applied to the texture feature learning of ultrasonic tumor classification in small data sets. Compared with other methods, the recognition rate of this algorithm is higher. This is because the proposed algorithm uses anisotropic diffusion denoising model to preprocess the data of ultrasonic images of the liver and spleen, which improves the image quality. Then, the external edge feature and the internal texture feature are taken as the double-input channels of the convolutional neural network, respectively, which makes the extracted features more abundant and conducive to recognition, so the recognition rate is higher. Based on the above results, it can be seen that the double-channel convolutional neural network can learn more features by using the two input channels of texture information and edge information of damaged images. Compared with other damage identification algorithms, double-channel CNN has higher accuracy in damage identification.


## 4.2.1. Validation of Proposed Double-Channel CNN Algorithm

In the convolutional layer of convolutional neural network, the effect of feature extraction of convolutional check with different sizes is different. In general, different sizes of convolutional kernels are set according to their characteristics when designing convolutional neural networks. After setting the appropriate size of the convolution kernel, the convolution layer can extract more sufficient and effective eigenvalues. Only when the feature extraction is sufficiently effective, can it provide effective guarantee for the subsequent output layer. In order to analyze the algorithm recognition rate of the proposed algorithm under different convolutional kernel sizes, the convolutional kernel size of the single-channel convolutional neural network was first set, and the feature extraction effect was observed. In the single-channel convolutional neural network A,3∗3, 5∗5, and 7∗7 are, respectively, set as the convolutional kernel in order to extract the texture information of the damaged site. In the single-channel convolutional neural network B, 3∗3, 5∗5, and 7∗7 are, respectively, set as convolutional cores in order to extract the edge information of the damaged site. The recognition results of single-channel convolutional neural network are shown in Table 2. It can be seen from Table 2 that in single-channel convolutional neural network A, the recognition result of 5∗5 convolutional kernel is higher; that is, its extraction of texture features is more effective. In single-channel convolutional neural network B, the recognition result of 7∗7 convolutional kernel is higher; that is, its extraction of edge information is better.Table 2
The recognition results of the double-channel and single-channel CNN.
Model structureLevel ILevel IILevel IIILevel IVLevel VSingle-channel CNN A (3∗3)81.2581.7482.3680.7480.53Single-channel CNN A (5∗5)83.8483.1783.4884.2585.3Single-channel CNN A (7∗7)82.3681.7682.8282.8580.94Single-channel CNN B (3∗3)80.1380.6281.2479.6279.41Single-channel CNN B (5∗5)81.3580.7581.8181.8479.93Single-channel CNN B (7∗7)82.6281.9582.2683.0384.08Double-channel CNN (5∗5 and 7∗7)94.9693.3394.6394.8292.40When setting the parameters of the two-channel convolutional neural network proposed in this paper,5∗5 convolutional kernel is selected as the convolutional kernel of the texture information input channel, and 7∗7 convolutional kernel is selected as the convolutional kernel of the edge information input channel. The results of the two-channel convolutional neural network in this paper are shown in the last column of Table 2. By comparing with single-channel convolutional neural network, it can be seen that the two-channel convolutional neural network algorithm proposed in this paper has a higher recognition rate.


## 4.2.2. Comparison Experiment

In order to compare the recognition performance of the algorithm presented in this paper, the damage recognition algorithms with excellent performance in recent years are selected for comparative experiments under the same experimental data. According to the experimental results above, the convolutional kernel size of the two-channel convolutional neural network designed in this paper is set as5∗5 and 7∗7. The damage recognition results of different algorithms are shown in Table 3.Table 3
The recognition results of different algorithms.
AlgorithmsLevel ILevel IILevel IIILevel IVLevel VLiterature [18]86.3382.5283.5284.2587.36Literature [19]88.4283.4186.2887.1382.74Literature [20]92.4191.9293.4991.3190.37Proposed method94.9693.3394.6394.8292.40Literature [18] proposed a method using multifractal dimension and back propagation neural network. Literature [19] used the deep learning framework to distinguish the unique types of acquired lesions and nodules of the breast through ultrasound imaging. Literature [20] proposed a stacked deep polynomial network (S-DPN) algorithm to improve the representation performance of the original DPN, and the S-DPN algorithm was applied to the texture feature learning of ultrasonic tumor classification in small data sets. Compared with other methods, the recognition rate of this algorithm is higher. This is because the proposed algorithm uses anisotropic diffusion denoising model to preprocess the data of ultrasonic images of the liver and spleen, which improves the image quality. Then, the external edge feature and the internal texture feature are taken as the double-input channels of the convolutional neural network, respectively, which makes the extracted features more abundant and conducive to recognition, so the recognition rate is higher. Based on the above results, it can be seen that the double-channel convolutional neural network can learn more features by using the two input channels of texture information and edge information of damaged images. Compared with other damage identification algorithms, double-channel CNN has higher accuracy in damage identification.


## 5. Conclusion and Future Work

In order to realize more accurate ultrasonic image diagnosis of liver and spleen injury, a double-channel convolutional neural network based algorithm for ultrasonic image classification diagnosis of liver and spleen injury is proposed. To improve the image quality of ultrasonic images, the anisotropic diffusion denoising model is first used to realize data preprocessing of ultrasonic images of the liver and spleen. The location of the lesion was detected with the outer edge to obtain the characteristics of the outer edge. Then, the rotation invariant local binary mode feature of the extracted image is taken as the inner texture feature of the image. In the process of convolutional neural network recognition, external edge features and internal texture features are used as double-input channels, respectively, so as to realize the effective use of the learning and recognition ability of convolutional neural network. The experiment results show that the method of this paper can accurately diagnose liver and spleen injury.In the future work, we will systematically analyze the time-consuming situation of each module algorithm, and put forward improvement measures to improve the overall efficiency of the algorithm.



---

*Source: 2827011-2021-07-05.xml*