# Artificial Intelligence-Assisted Visual Sensing Technology under Duodenoscopy of Gallbladder Stones

Authors: Dongyan Li; Bin Du; Youhui Shen; Lin Ge
Journal: Journal of Sensors
 (2021)

---

## Abstract

Objective. This study is aimed at exploring the application effect of duodenoscopy assisted by visual sensing technology based on convolutional neural network (CNN) segmentation algorithm in the diagnosis and treatment of gallbladder stones, so as to provide safer and more effective treatment methods for patients with gallstones. Methods. 188 patients with gallstones and choledocholithiasis who were admitted to our hospital from January 2016 to April 2021 were selected as the research objects. Based on whether the patients were willing to use AI-assisted visual sensing technology during the treatment process, all patients were divided into two groups, namely, the AI group and the conventional group. Various surgical indicators of patients in two groups were compared. Results. The precision, recall, and mean intersection ratio of the M-Unet-based segmentation algorithm were 94.56%, 96.56%, and 98.92%, respectively. In the AI group, the operation time (2.74±0.45h), postoperative drainage tube placement time (4.31±1.15d), time required for recovery of gastrointestinal function (1.74±0.54d), time required to get out of bed (1.14±0.55h), and time spent in hospital (9.94±1.45d) were all shorter compared with those in the conventional group, which were 3.21±0.32h, 12.14±2.98d, 2.89±0.67d, 2.09±0.87h, and 14.14±1.15h, showing statistical differences (P<0.05); the intraoperative blood loss (79.74±6.45mL) and residual status of stones (0%) in the AI group were much lower than those in the conventional group (P<0.05). In addition, the incidence of complications (10.26%) and the indicators of postoperative gallbladder function of patients in the AI group were lower greatly than those in the conventional group (P<0.05). Conclusion. The visual sensing technology assisted by the CNN algorithm showed a good effect on image processing, and endoscopic technology can effectively improve the treatment effect of gallbladder stones combined with choledocholithiasis with the aid of this technology. Therefore, the conclusion in this study proved that visual sensing technology based on intelligent algorithms showed a good future in the medical field.


---

## Body

## 1. Introduction

The current society is in an era of rapid development of technology and civilization. People’s living standards and quality of life have been greatly improved, which is a beneficial aspect of social development. However, it is not only the beneficial side that develops along with it, for example, various diseases also closely follow. Nowadays, people do not pay enough attention to dietary hygiene and dietary rules, so the probability of occurrence of digestive system diseases is becoming higher and higher. These digestive system diseases include gallbladder and biliary system diseases, especially stone diseases of the gallbladder and biliary tract [1, 2]. According to relevant research statistics, the incidence of gallbladder and biliary calculus diseases is basically about 10% among the adult population in China; and in this type of disease, the probability of patients with gallbladder stones combined with choledocholithiasis is about 12%~15% [3]. Therefore, a large number of clinical applications have been done to provide patients with gallbladder stones combined with choledocholithiasis with more convenient and effective treatment methods. After continuous efforts in researches of many experts and researchers, endoscopic technology has been introduced into the clinical treatment of gallbladder stones and has been widely applied in clinical medicine today. However, laparoscopic technology and duodenoscopy are often used clinically in the treatment of this disease due to the complicated condition of gallbladder stones combined with choledocholithiasis [4]. A duodenoscope is mostly used to guide the implementation of bile duct stone removal. It refers to the method of pulling the bile duct stones out of the bile duct using net baskets, balloons, and other instruments under a duodenoscope. The main indications are choledocholithiasis, hepatolithiasis, and gallbladder stones [5]. Laparoscopy is mostly used to guide gallbladder resection. The main operation method is to insert a special catheter from the abdomen into the peritoneal cavity and then infuse about 2~5 L of carbon dioxide into the peritoneal cavity. When the abdominal pressure reaches a certain level, three incisions of 0.5~1.5 cm are made in the abdomen to dissect the triangle structure of the gallbladder. Then, the cystic duct and cystic artery are separated and clamped, and then, the entire gallbladder including the stones is resected [6, 7]. Although the treatment effect has made people feel satisfied, there is still a big gap between the effect observed through the display mirror and the effect under direct eyes. With the rapid development of science and technology these years, the artificial intelligence (AI) technology has been widely used in various fields, among which AI and visual sensing technology have developed rapidly. The main function of this technology is to add perception technology to the visual sensor so that it can segment, detect, and recognize the network architecture of the target object [8]. The basic theory of this technology is the convolutional neural network (CNN) algorithm [9], and visual sensing technology assisted by the CNN system has gradually reached or exceeded the recognition level of humans. In the past, this technology was mostly used for segmentation and identification of commodity goods [10] and measurement of pitting of gears in shooting ranges [11]. Its application in the field of medical examination is relatively small. To provide a safer and more effective treatment for patients with gallstones, the patients with gallstones and choledocholithiasis were selected as the research objects in this study. In addition, the laparoscopic technology, duodenoscopy, and visual sensing technology assisted by CNN segmentation algorithm were combined for research and analysis of the treatment of gallbladder stones combined with common bile duct stones, hoping that this method can give research basis for clinical treatment.


## 2. Materials and Methods

### 2.1. Visual Sensing Technology Based on CNN

The composition of the CNN system is based on the traditional neural network. CNN is a recognition process that gradually develops from a part to a whole in the aspect of biometric recognition [12]. Compared with the traditional neural networks [13, 14], the CNN system shows excellent features such as local receptive fields, shared weights, and multicore convolution. The existence of above advantages can not only greatly reduce the parameters in the neural network but also obtain different levels of characteristic information through the hierarchical network structure. The specific structure of the CNN system is shown in Figure 1. Its main components can be divided into a convolutional layer, an activation function, a convergence layer, and a fully connected layer. Its own parameters were optimized using an iterative optimization algorithm, and the network structure was converged, so as to further achieve the convergence of target loss function. Next, each layer of CNN would be explained and described in detail.Figure 1
The specific structure of the CNN system.The convolutional layer is a type of local connection, which can reduce the number of model parameters. Each network neuron in the convolutional layer will only be connected to the local receptive field of the previous layer. However, the size of the local receptive field is mainly determined by the parameter size of the convolution kernel. The representative way of the two-dimensional convolution in the convolutional layer in CNN was as follows:(1)si,j=Q∗Wi,j=∑m∑nQi+m,j+nWm,n.In the above equation,si,j referred to result of the convolution of the target image; Q∗W was multiplication of the local area Q of the target image and the elements of each position in the convolution kernel W matrix; i,j represented the pixel at the target position of the image; and m and n referred to the number of parameters of the local area Q and the convolution kernel W of the target image, respectively.The activation layer is generally connected after the convolutional layer, and the data is nonlinearly transformed by the activation function in the activation layer. The more commonly used activation functions include sigmoid activation function, tanh activation function, and ReLU activation function. The specific expressions of these three activation functions were as follows:(2)fx=11+e−x,(3)fx=tanhx,(4)fx=maxx,0.In the expression equations of the above three activation functions, the ReLU activation function with relatively good effect was adopted in this study.The main function of the pooling layer itself is to perform lower-level sampling on the feature map, so as to reduce the parameters required for calculation and overfitting and further improve the stability of CNN. There were two main pooling methods adopted in this study, namely, regional maximum pooling and average pooling.The Softmax classifier belongs to a multinomial logistic regression model, which can be classified as a log-linear model. Most of the time, it is used in binary classification. The specific calculation process of the Softmax classifier can be expressed as follows:(5)PYx=pwk⋅x1+∑k=1K−1pwk⋅x,k=1,2,3,⋯,K−1,(6)PYx=11+∑k=1K−1pwk⋅x,k=K,(7)PYx=pwk⋅x∑k=1K−1pwk⋅x,k=1,2,3,⋯,K.In the above three equations,W referred to the required parameter model; q was an input vector; PY∣q referred to the probability of predicting the category of q as category Y, and the possible value of Y was k=1,2,3,⋯,K. The dataset required for training was given as U=x1,y1,x2,y2,⋯xm,ym (xi∈Rn and yi∈1,2,⋯,K), and yi and xi corresponded to the category label. The maximum likelihood estimation method was adopted in this study to calculate the result of the model. Therefore, the function of xi could be expressed as
(8)ℝj=1K1Y=jPY=jxi;w=ℝj=1K1Y=jpwjU⋅x∑k=1KpwjU⋅x.Then, the expression of its logarithmic function was given as follows:(9)∑jk1Y=jlogpwjU⋅x∑k=1KpwjU⋅x.The expression of1withtruevaluewas=1, and the expression of 1withfalsevaluewas=0. For the entire dataset U, its log likelihood function was written as follows:
(10)1m∑i=1m∑j=1K1yi=jlogpwjU⋅x∑k=1KpwjU⋅x.Maximizing the log-likelihood function was equivalent to minimizing the loss functionJw:
(11)Jw=1m∑i=1m∑j=1K1yi=jlogpwjU⋅x∑k=1KpwjU⋅x.In equation (11), when the value of Jw was the smallest, w was the model parameter to be sought; and Jw was a convex function. Then, the method was optimized to make the result converge to the best in the whole process.Adam optimizer [15] combines the advantages of two types of optimization algorithms: AdaGrad optimization algorithm and RMSProp optimization algorithm. The optimization method is to use the dynamic changes of the first-order moment estimation (the average of all gradients) and the second-order moment estimation (the variance of the gradient without centering) to adjust each parameter of learning rate.Then, the expressions of the first-order momentmu and the second-order moment ou of the u-th iteration are shown in
(12)mu=αmu−1+1−αgu,(13)ou=βou−1+1−βgu2.In the above two equations,α and β were parameter values, which were defaulted to 0.9 and 0.99, respectively. Then, the required unbiased estimation of the first and second moments required proper processing of mu and ou. The specific processing process was as follows:
(14)mu′=mu1−αu,(15)ou′=ou1−βu.After further improvement, the manifestation of the parameters can be expressed as below equation:(16)χu=−δou′+εmu′.After Adam was corrected by equations (14) and (15), the range of the learning rate of each iteration can be determined, and the parameters were relatively stable.The propagation algorithm in CNN can be classified into forward propagation algorithm and backward propagation algorithm.The forward propagation algorithm [16] refers to the process in which information starts from the input layer and then propagates forward in a one-way propagation manner, penetrating the neural network to the output layer, and finally being output by the network. The information output from the h−1-th layer in the overpropagation process was the input information of the h-th layer. It was supposed that Zh was the output information of the h−1-th layer, where h=1,2,3,⋯,H.It was assumed that theh-th layer was a convolutional layer, and then, the algorithm at this layer can be expressed as follows:
(17)Zh=fh−1WhZh−1+bh.If theh-th layer was assumed as the pooling layer, then the algorithm at this layer can be written as follows:
(18)Zh=PZh−1.If theh-th layer was assumed as the output layer, then the algorithm at this layer can be written as follows:
(19)ZH=CWhZh−1+bh.In equations (17)–(19), Wh and bh represented the weight and bias of the h-th layer, respectively; fh−1 was the activation function in the h−1-th layer, which was the ReLU activation function mentioned above; P referred to the process of pooling the input through the fixed pooling function; and C was the Softmax classifier mentioned above.The backpropagation algorithm [17, 18] refers to propagating the residuals backwards during the training process and then gradually correcting the parameters in the CNN, which greatly simplifies the solution of the partial derivatives of the variables in the multilayer composite function. The specific calculation method was shown in equations (20)–(25).The expression equation of the loss function can be assumed as follows:(20)JW,b;x,y=12aW,bx−y2.The variables in equation (20) were specifically expressed as follows. x was an input information; W and b represented the parameters and bias of the network model, respectively; aW,bx−y referred to the probability that the network predicted the input x as the category y. The activation value was calculated based on the forward propagation algorithm, and then, the residual error ∂ih of the i-th neuron node in the h-th layer was calculated. The specific calculation equation was expressed as follows.
(21)∂ih=κκdihJW,b;x,y.In the equation above,dih referred to i-th unit of the h-th layer, including the weighted sum of the bias units, and had not been processed by the activation function.If theh-th layer was a convolutional layer, then the h+1-th layer was the pooling layer, and the residual calculation could be realized with below equation:
(22)∂ih=upsample∂ih+1⊙f′dih.In equation (22), ⊙ referred to the dot product operation of matrix, upsample was to restore the residual matrix to before pooling, and f′dih was the derivative value of activation function f to dh.If theh-th layer was a pooling layer, the h+1-th layer was a convolutional layer, and the residual could be calculated with
(23)∂ih=∂ih+1∗rot180Wh+1⊙f′dih.In the equation above,∗ was convolution, and rot180 referred to flipping up and down once and then flipping once left and right.Finally, the partial derivatives of each layer could be written as(24)κκWijhJW,b;x,y=∂ih+1Zjh,(25)κκbihJW,b;x,y=∂ih+1.A multipath dilated convolution (MPDC) [19] is proposed to further optimize the CNN image segmentation algorithm. The MPDC-Unet (M-Unet) was adopted for description in the following text (Figure 2), and the Soft Dice loss was applied for program supervision. In this network, its overall architecture was established in the way of encoding and decoding. It should encode (i.e., downsampling) and decode (i.e., upsampling) firstly and then get the segmentation result mask with the same size as the initial image. The encoding process used Max Pooling for double downsampling, and multiple convolutions were used between the two pooling layers to encode features. In the decoding stage, deconvolution was to gradually increase the resolution of the feature map, and the large convolution kernel was used to encode the context information. The low-level output feature map was firstly cropped to make it the same spatial size as the corresponding high-level feature map and then spliced in the channel dimension to achieve feature fusion so that the low-level and high-level features can be fused better.Figure 2
Module diagram of MPDC (the number in the arrow indicated the void ratio).The most commonly used loss function in semantic segmentation [20] is the pixel-wise cross-entropy loss. For each pixel, the predicted category was compared with the true value category, and the loss of all pixels was averaged, that was, the loss of each pixel in the image was treated equally. In this study, the Soft Dice coefficient was undertaken as the loss function, which could be expressed as follows:
(26)SoftDice=2∑iMpigi∑iMpi2+∑iMgi2.In the equation above,pi and gi represented the value of the prediction mask and the true value mask, respectively. The cross-entropy loss function paid more attention to the global pixel classification including the background and foreground, which exerted a better effect on reducing missegmentation. Therefore, the Soft Dice loss and cross-entropy weighted summation were adopted in this study to replace the traditional cross-entropy loss function in M-Unet as shown in
(27)V=w1×SoftDice+w2×CrossEntropy.In the above equation, when the values ofw1 and w2 were both 0.5, the result was the best. The main process of image segmentation based on the M-Unet network model is shown in Figure 3.Figure 3
The main process of image segmentation based on the M-Unet network model.The indicators commonly used to evaluate the effect of semantic segmentation included precision, recall, and mean intersection ratio. If there were two sets of pixel sets for the research target, then the actual set GT and the predicted set P can be defined as follows. The first referred to the correctly predicted pixels (true positive, TP), and TP referred to the intersection of the two sets; the second was false positive (FP), which referred to the predicted wrong pixel in the predicted pixel set; and the last was false negative (FN), which referred to the wrongly predicted pixels in the set of true value pixels. According to the above definitions, the higher the precision (that was, the ratio of the predicted correct pixel to the predicted total pixel), the higher the FP was (the fewer false predictions). The precision can be calculated with equation (28). The higher the recall (the ratio of the predicted correct pixel to the true value pixel) was, the less FN was, which meant the less missed detection of segmentation result. It can be calculated with equation (29). mIoU was the intersection ratio of two sets, which was used to judge the coincidence rate between the prediction result and the target true value. The higher the mean intersection ratio, the more accurate the segmentation algorithm. Its calculation expression was given as equation (30).
(28)Pre=TPTP+FP,(29)Rec=TPTP+FN,(30)mIoU=TPTP+FP+FN.In the above three equations, Pre, Rec, and mIoU referred to precision, recall, and mean intersection ratio, respectively.


### 2.2. Research Objects

The patients who were diagnosed with gallbladder stones combined with choledocholithiasis during January 2016 to April 2021 were randomly selected as the research objects with a total of 188 cases. Then, based on whether all patients were willing to accept the AI-assisted and visual sensing technology during the treatment process, all patients were divided into two groups: AI group and conventional group. According to statistics, there were 78 patients in the AI group, of which 38 were males and 40 were females. They were all between 26 and 75 years old, with an average age of42.72±2.87 years; and the course of disease was 1.17~5.67 months with the average value of 3.37±0.78 months; and the number of stones was 17 cases of single stones and 61 cases of multiple stones. There were 110 patients in the conventional group, including 51 males and 59 females; they were aged 24~74 years with the average age of 42.45±3.12 years; the disease course was also 1.16~5.68 months (with an average course of the disease of 3.44±0.12 months); and there were 26 cases of single stones and 84 cases of multiple stones. The research had been applied to the medical ethics committee of our hospital, and the research permission had been obtained.The inclusion criteria were defined as follows: (a) all patients had been diagnosed with gallbladder stones combined with choledocholithiasis by CT or B-ultrasound and suffered from clinical manifestations such as jaundice, abdominal pain, nausea, and vomiting before the surgery; (b) after diagnosis, all patients were found to have indications for laparoscopy and duodenoscope diagnosis and treatment; (c) routine hematuria examinations of all patients showed that they were normal, and they could follow the doctor’s instructions well; and (d) all patients and their family members had signed the informed consent forms.The exclusion criteria were determined as follows: (a) patients with severe liver, kidney, and other organ failure diseases and endocrine diseases; patients who had taken drugs that may affect the research indicators within one month before surgery; patients who did not cooperate or comply during treatment and who did not cooperate with follow-up after surgery; patients who also received other treatment methods during treatment; and patients whose condition suddenly worsens or was transferred to hospital during treatment.


### 2.3. Treatment Methods

All patients were intubated in a supine position under general anesthesia. Then, the laparoscope was placed, and the specific operations were as follows. Firstly, 1 cm below the belly button was undertaken as the entrance of the laparoscope to make a transverse cut here. The incision was about 2~3 cm. Then, the abdomen was inflated and the pressure was maintained at 12~14 mmHg to separate the organs from the abdominal tissues and fully expose them. Under the above circumstances, the size and position of the gallbladder in the abdominal cavity and its adhesion to the surrounding tissues were observed. All patients were treated by laparoscopy combined with duodenoscope. The duodenoscope was inserted from the oral cavity, the duodenal papilla was found according to the duodenoscope, and the guide wire was inserted into the common bile duct along the nipple. During the surgery, patients in the conventional group were performed with endoscopy for retrograde cholangiopancreatography, while patients in the AI group received the visual sensor and the endoscope to directly observe the gallbladder and bile duct. The location and size of the stones in the bile duct were observed. Afterwards, the papillary sphincter was incised (when the incision was about 1 cm and the stone was less than 1 cm, the balloon was to directly remove the stone; when the stone was larger than 1 cm, it had to be crushed firstly before removing). After the stone removal, it was observed again. If it was observed that the stones in the common bile duct had been removed, a plastic stent of the bile duct was placed for nasal drainage, and the patient’s liver function and amylase indicators were monitored synchronously. After 3 days, laparoscopic cholecystectomy was performed for surgical treatment. The specific process is shown in Figure4. Two weeks after the patient was discharged from the hospital, the biliary stent was removed by a duodenoscope, and the stone removal was observed by the above two methods.Figure 4
Laparoscopic cholecystectomy process.


### 2.4. Evaluation Indicators

The blood loss, operation time, postoperative drainage tube placement time, gastrointestinal function recovery, extrabed activity time, postoperative complications, residual stones, and time spent in hospital for the two groups of patients were observed and recorded during the surgery.The total serum bilirubin, direct bilirubin, alanine aminotransferase, and alkaline phosphatase were measured with the Romer combined biochemical analyzer (Roche Diagnostics (Shanghai) Co., Ltd. COBAS INTEGRA 800) and compared.


### 2.5. Statistical Analysis

In this study, SPSS 25.0 was applied to calculate the data. The count data was expressed in the form of percentage, and theχ2 test was used. The measurement data was expressed in the form of x±s, and the t-test was used. P<0.05 was considered that the difference was statistically significant.


## 2.1. Visual Sensing Technology Based on CNN

The composition of the CNN system is based on the traditional neural network. CNN is a recognition process that gradually develops from a part to a whole in the aspect of biometric recognition [12]. Compared with the traditional neural networks [13, 14], the CNN system shows excellent features such as local receptive fields, shared weights, and multicore convolution. The existence of above advantages can not only greatly reduce the parameters in the neural network but also obtain different levels of characteristic information through the hierarchical network structure. The specific structure of the CNN system is shown in Figure 1. Its main components can be divided into a convolutional layer, an activation function, a convergence layer, and a fully connected layer. Its own parameters were optimized using an iterative optimization algorithm, and the network structure was converged, so as to further achieve the convergence of target loss function. Next, each layer of CNN would be explained and described in detail.Figure 1
The specific structure of the CNN system.The convolutional layer is a type of local connection, which can reduce the number of model parameters. Each network neuron in the convolutional layer will only be connected to the local receptive field of the previous layer. However, the size of the local receptive field is mainly determined by the parameter size of the convolution kernel. The representative way of the two-dimensional convolution in the convolutional layer in CNN was as follows:(1)si,j=Q∗Wi,j=∑m∑nQi+m,j+nWm,n.In the above equation,si,j referred to result of the convolution of the target image; Q∗W was multiplication of the local area Q of the target image and the elements of each position in the convolution kernel W matrix; i,j represented the pixel at the target position of the image; and m and n referred to the number of parameters of the local area Q and the convolution kernel W of the target image, respectively.The activation layer is generally connected after the convolutional layer, and the data is nonlinearly transformed by the activation function in the activation layer. The more commonly used activation functions include sigmoid activation function, tanh activation function, and ReLU activation function. The specific expressions of these three activation functions were as follows:(2)fx=11+e−x,(3)fx=tanhx,(4)fx=maxx,0.In the expression equations of the above three activation functions, the ReLU activation function with relatively good effect was adopted in this study.The main function of the pooling layer itself is to perform lower-level sampling on the feature map, so as to reduce the parameters required for calculation and overfitting and further improve the stability of CNN. There were two main pooling methods adopted in this study, namely, regional maximum pooling and average pooling.The Softmax classifier belongs to a multinomial logistic regression model, which can be classified as a log-linear model. Most of the time, it is used in binary classification. The specific calculation process of the Softmax classifier can be expressed as follows:(5)PYx=pwk⋅x1+∑k=1K−1pwk⋅x,k=1,2,3,⋯,K−1,(6)PYx=11+∑k=1K−1pwk⋅x,k=K,(7)PYx=pwk⋅x∑k=1K−1pwk⋅x,k=1,2,3,⋯,K.In the above three equations,W referred to the required parameter model; q was an input vector; PY∣q referred to the probability of predicting the category of q as category Y, and the possible value of Y was k=1,2,3,⋯,K. The dataset required for training was given as U=x1,y1,x2,y2,⋯xm,ym (xi∈Rn and yi∈1,2,⋯,K), and yi and xi corresponded to the category label. The maximum likelihood estimation method was adopted in this study to calculate the result of the model. Therefore, the function of xi could be expressed as
(8)ℝj=1K1Y=jPY=jxi;w=ℝj=1K1Y=jpwjU⋅x∑k=1KpwjU⋅x.Then, the expression of its logarithmic function was given as follows:(9)∑jk1Y=jlogpwjU⋅x∑k=1KpwjU⋅x.The expression of1withtruevaluewas=1, and the expression of 1withfalsevaluewas=0. For the entire dataset U, its log likelihood function was written as follows:
(10)1m∑i=1m∑j=1K1yi=jlogpwjU⋅x∑k=1KpwjU⋅x.Maximizing the log-likelihood function was equivalent to minimizing the loss functionJw:
(11)Jw=1m∑i=1m∑j=1K1yi=jlogpwjU⋅x∑k=1KpwjU⋅x.In equation (11), when the value of Jw was the smallest, w was the model parameter to be sought; and Jw was a convex function. Then, the method was optimized to make the result converge to the best in the whole process.Adam optimizer [15] combines the advantages of two types of optimization algorithms: AdaGrad optimization algorithm and RMSProp optimization algorithm. The optimization method is to use the dynamic changes of the first-order moment estimation (the average of all gradients) and the second-order moment estimation (the variance of the gradient without centering) to adjust each parameter of learning rate.Then, the expressions of the first-order momentmu and the second-order moment ou of the u-th iteration are shown in
(12)mu=αmu−1+1−αgu,(13)ou=βou−1+1−βgu2.In the above two equations,α and β were parameter values, which were defaulted to 0.9 and 0.99, respectively. Then, the required unbiased estimation of the first and second moments required proper processing of mu and ou. The specific processing process was as follows:
(14)mu′=mu1−αu,(15)ou′=ou1−βu.After further improvement, the manifestation of the parameters can be expressed as below equation:(16)χu=−δou′+εmu′.After Adam was corrected by equations (14) and (15), the range of the learning rate of each iteration can be determined, and the parameters were relatively stable.The propagation algorithm in CNN can be classified into forward propagation algorithm and backward propagation algorithm.The forward propagation algorithm [16] refers to the process in which information starts from the input layer and then propagates forward in a one-way propagation manner, penetrating the neural network to the output layer, and finally being output by the network. The information output from the h−1-th layer in the overpropagation process was the input information of the h-th layer. It was supposed that Zh was the output information of the h−1-th layer, where h=1,2,3,⋯,H.It was assumed that theh-th layer was a convolutional layer, and then, the algorithm at this layer can be expressed as follows:
(17)Zh=fh−1WhZh−1+bh.If theh-th layer was assumed as the pooling layer, then the algorithm at this layer can be written as follows:
(18)Zh=PZh−1.If theh-th layer was assumed as the output layer, then the algorithm at this layer can be written as follows:
(19)ZH=CWhZh−1+bh.In equations (17)–(19), Wh and bh represented the weight and bias of the h-th layer, respectively; fh−1 was the activation function in the h−1-th layer, which was the ReLU activation function mentioned above; P referred to the process of pooling the input through the fixed pooling function; and C was the Softmax classifier mentioned above.The backpropagation algorithm [17, 18] refers to propagating the residuals backwards during the training process and then gradually correcting the parameters in the CNN, which greatly simplifies the solution of the partial derivatives of the variables in the multilayer composite function. The specific calculation method was shown in equations (20)–(25).The expression equation of the loss function can be assumed as follows:(20)JW,b;x,y=12aW,bx−y2.The variables in equation (20) were specifically expressed as follows. x was an input information; W and b represented the parameters and bias of the network model, respectively; aW,bx−y referred to the probability that the network predicted the input x as the category y. The activation value was calculated based on the forward propagation algorithm, and then, the residual error ∂ih of the i-th neuron node in the h-th layer was calculated. The specific calculation equation was expressed as follows.
(21)∂ih=κκdihJW,b;x,y.In the equation above,dih referred to i-th unit of the h-th layer, including the weighted sum of the bias units, and had not been processed by the activation function.If theh-th layer was a convolutional layer, then the h+1-th layer was the pooling layer, and the residual calculation could be realized with below equation:
(22)∂ih=upsample∂ih+1⊙f′dih.In equation (22), ⊙ referred to the dot product operation of matrix, upsample was to restore the residual matrix to before pooling, and f′dih was the derivative value of activation function f to dh.If theh-th layer was a pooling layer, the h+1-th layer was a convolutional layer, and the residual could be calculated with
(23)∂ih=∂ih+1∗rot180Wh+1⊙f′dih.In the equation above,∗ was convolution, and rot180 referred to flipping up and down once and then flipping once left and right.Finally, the partial derivatives of each layer could be written as(24)κκWijhJW,b;x,y=∂ih+1Zjh,(25)κκbihJW,b;x,y=∂ih+1.A multipath dilated convolution (MPDC) [19] is proposed to further optimize the CNN image segmentation algorithm. The MPDC-Unet (M-Unet) was adopted for description in the following text (Figure 2), and the Soft Dice loss was applied for program supervision. In this network, its overall architecture was established in the way of encoding and decoding. It should encode (i.e., downsampling) and decode (i.e., upsampling) firstly and then get the segmentation result mask with the same size as the initial image. The encoding process used Max Pooling for double downsampling, and multiple convolutions were used between the two pooling layers to encode features. In the decoding stage, deconvolution was to gradually increase the resolution of the feature map, and the large convolution kernel was used to encode the context information. The low-level output feature map was firstly cropped to make it the same spatial size as the corresponding high-level feature map and then spliced in the channel dimension to achieve feature fusion so that the low-level and high-level features can be fused better.Figure 2
Module diagram of MPDC (the number in the arrow indicated the void ratio).The most commonly used loss function in semantic segmentation [20] is the pixel-wise cross-entropy loss. For each pixel, the predicted category was compared with the true value category, and the loss of all pixels was averaged, that was, the loss of each pixel in the image was treated equally. In this study, the Soft Dice coefficient was undertaken as the loss function, which could be expressed as follows:
(26)SoftDice=2∑iMpigi∑iMpi2+∑iMgi2.In the equation above,pi and gi represented the value of the prediction mask and the true value mask, respectively. The cross-entropy loss function paid more attention to the global pixel classification including the background and foreground, which exerted a better effect on reducing missegmentation. Therefore, the Soft Dice loss and cross-entropy weighted summation were adopted in this study to replace the traditional cross-entropy loss function in M-Unet as shown in
(27)V=w1×SoftDice+w2×CrossEntropy.In the above equation, when the values ofw1 and w2 were both 0.5, the result was the best. The main process of image segmentation based on the M-Unet network model is shown in Figure 3.Figure 3
The main process of image segmentation based on the M-Unet network model.The indicators commonly used to evaluate the effect of semantic segmentation included precision, recall, and mean intersection ratio. If there were two sets of pixel sets for the research target, then the actual set GT and the predicted set P can be defined as follows. The first referred to the correctly predicted pixels (true positive, TP), and TP referred to the intersection of the two sets; the second was false positive (FP), which referred to the predicted wrong pixel in the predicted pixel set; and the last was false negative (FN), which referred to the wrongly predicted pixels in the set of true value pixels. According to the above definitions, the higher the precision (that was, the ratio of the predicted correct pixel to the predicted total pixel), the higher the FP was (the fewer false predictions). The precision can be calculated with equation (28). The higher the recall (the ratio of the predicted correct pixel to the true value pixel) was, the less FN was, which meant the less missed detection of segmentation result. It can be calculated with equation (29). mIoU was the intersection ratio of two sets, which was used to judge the coincidence rate between the prediction result and the target true value. The higher the mean intersection ratio, the more accurate the segmentation algorithm. Its calculation expression was given as equation (30).
(28)Pre=TPTP+FP,(29)Rec=TPTP+FN,(30)mIoU=TPTP+FP+FN.In the above three equations, Pre, Rec, and mIoU referred to precision, recall, and mean intersection ratio, respectively.


## 2.2. Research Objects

The patients who were diagnosed with gallbladder stones combined with choledocholithiasis during January 2016 to April 2021 were randomly selected as the research objects with a total of 188 cases. Then, based on whether all patients were willing to accept the AI-assisted and visual sensing technology during the treatment process, all patients were divided into two groups: AI group and conventional group. According to statistics, there were 78 patients in the AI group, of which 38 were males and 40 were females. They were all between 26 and 75 years old, with an average age of42.72±2.87 years; and the course of disease was 1.17~5.67 months with the average value of 3.37±0.78 months; and the number of stones was 17 cases of single stones and 61 cases of multiple stones. There were 110 patients in the conventional group, including 51 males and 59 females; they were aged 24~74 years with the average age of 42.45±3.12 years; the disease course was also 1.16~5.68 months (with an average course of the disease of 3.44±0.12 months); and there were 26 cases of single stones and 84 cases of multiple stones. The research had been applied to the medical ethics committee of our hospital, and the research permission had been obtained.The inclusion criteria were defined as follows: (a) all patients had been diagnosed with gallbladder stones combined with choledocholithiasis by CT or B-ultrasound and suffered from clinical manifestations such as jaundice, abdominal pain, nausea, and vomiting before the surgery; (b) after diagnosis, all patients were found to have indications for laparoscopy and duodenoscope diagnosis and treatment; (c) routine hematuria examinations of all patients showed that they were normal, and they could follow the doctor’s instructions well; and (d) all patients and their family members had signed the informed consent forms.The exclusion criteria were determined as follows: (a) patients with severe liver, kidney, and other organ failure diseases and endocrine diseases; patients who had taken drugs that may affect the research indicators within one month before surgery; patients who did not cooperate or comply during treatment and who did not cooperate with follow-up after surgery; patients who also received other treatment methods during treatment; and patients whose condition suddenly worsens or was transferred to hospital during treatment.


## 2.3. Treatment Methods

All patients were intubated in a supine position under general anesthesia. Then, the laparoscope was placed, and the specific operations were as follows. Firstly, 1 cm below the belly button was undertaken as the entrance of the laparoscope to make a transverse cut here. The incision was about 2~3 cm. Then, the abdomen was inflated and the pressure was maintained at 12~14 mmHg to separate the organs from the abdominal tissues and fully expose them. Under the above circumstances, the size and position of the gallbladder in the abdominal cavity and its adhesion to the surrounding tissues were observed. All patients were treated by laparoscopy combined with duodenoscope. The duodenoscope was inserted from the oral cavity, the duodenal papilla was found according to the duodenoscope, and the guide wire was inserted into the common bile duct along the nipple. During the surgery, patients in the conventional group were performed with endoscopy for retrograde cholangiopancreatography, while patients in the AI group received the visual sensor and the endoscope to directly observe the gallbladder and bile duct. The location and size of the stones in the bile duct were observed. Afterwards, the papillary sphincter was incised (when the incision was about 1 cm and the stone was less than 1 cm, the balloon was to directly remove the stone; when the stone was larger than 1 cm, it had to be crushed firstly before removing). After the stone removal, it was observed again. If it was observed that the stones in the common bile duct had been removed, a plastic stent of the bile duct was placed for nasal drainage, and the patient’s liver function and amylase indicators were monitored synchronously. After 3 days, laparoscopic cholecystectomy was performed for surgical treatment. The specific process is shown in Figure4. Two weeks after the patient was discharged from the hospital, the biliary stent was removed by a duodenoscope, and the stone removal was observed by the above two methods.Figure 4
Laparoscopic cholecystectomy process.


## 2.4. Evaluation Indicators

The blood loss, operation time, postoperative drainage tube placement time, gastrointestinal function recovery, extrabed activity time, postoperative complications, residual stones, and time spent in hospital for the two groups of patients were observed and recorded during the surgery.The total serum bilirubin, direct bilirubin, alanine aminotransferase, and alkaline phosphatase were measured with the Romer combined biochemical analyzer (Roche Diagnostics (Shanghai) Co., Ltd. COBAS INTEGRA 800) and compared.


## 2.5. Statistical Analysis

In this study, SPSS 25.0 was applied to calculate the data. The count data was expressed in the form of percentage, and theχ2 test was used. The measurement data was expressed in the form of x±s, and the t-test was used. P<0.05 was considered that the difference was statistically significant.


## 3. Results

### 3.1. Image Segmentation Results

Figure5 shows the captured images of the gallbladder stones and choledocholithiasis under laparoscopy and duodenoscope. Figures 5(a)–5(d) are the initial images under the endoscopy, and Figures 5(a1)–5(d1) are the images under the M-Unet segmentation. Figure 5(a) is a mirror image of the complete gallbladder of the abdominal cavity. It showed that the diameter of the gallbladder was slightly increased, but there was no inflammatory response on the surface. Figure 5(a1) shows the shape of the gallbladder segmented by M-Unet in the original image which performed. Figure 5(b) is an image of the bile duct after the surface mucosal tissue had been removed under laparoscopic surgery, and the shape of the bile duct can be clearly observed where the yellow arrow pointed. Figure 5(b1) shows the morphological performance of the bile duct after M-Unet segmentation. Figure 5(c) shows the state of stones in the bile duct observed under the duodenum meridian, as pointed by the yellow arrow. It can be concluded that the surface of the stones was smooth, and the overall colour was black. Compared with the diameter of the bile duct, the diameter of the stones was relatively small. Figure 5(c1) shows the morphological performance of bile duct stones obtained by M-Unet segmentation. Observation suggested that only the segmentation was the part shown by the plain film. Figure 5(d) shows a stone blocked in the bile duct orifice under a duodenoscope (pointed by the yellow arrow). It can be observed that the bile duct orifice had been cracked due to the blockage of the stone, and inflammation was visible. It also indirectly showed that the diameter of the stones in the figure was larger than the diameter of the bile duct, and the stones had to be crushed before taking them. Figure 5(d1) shows the stone part of the bile duct orifice segmented by M-Unet. After comparative analysis, the segmentation was relatively complete, and there was basically no missing segmentation. After calculation, the precision, recall, and the mean intersection ratio of the M-Unet-based segmentation algorithm were 94.56%, 96.56%, and 98.92%, respectively. The results were quite good and can be used in this study.Figure 5
The captured images of the gallbladder stones and choledocholithiasis under laparoscopy and duodenoscope. Note: (a) and (a1) show the images of the gallbladder; (b) and (b1) show the images of the bile duct; (c) and (d) show the images of the common bile duct; and (c1) and (d1) show the stone in the common bile duct. The yellow arrows on images (b), (c), and (d) referred to the cystic duct, stones in the common bile duct, and stones in the common bile duct, respectively.


### 3.2. Comparison on General Data of Patients

According to the research statistics, Figure6 shows the general treatment status of the two groups of patients in this study, including gender distribution, age distribution, average course of disease, and distribution of the number of stones. In terms of gender distribution, the proportion of men in the AI group was 48.72% (38/78) and the proportion of women was 51.28% (40/78); the proportions of men and women in the conventional group were 46.36% (51/110) and 53.64% (59/110), respectively (Figure 6(a)). In terms of age, the patients were divided into five stages: 25–35, 36–45, 46–55, 56–65, and >66 years old. The proportions of patients in five stages in the AI group were 8.97% (7/78), 19.23% (15/78), 29.49% (23/78), 15.38% (12/78), and 26.92% (21/78), respectively; while those in the conventional group were 7.27% (8/110), 21.82% (24/110), 24.55% (27/110), 18.18% (20/110), and 28.18% (31/110), respectively (Figure 6(b)). The average course of disease for patients in the AI group was 3.37±0.78 months, and that in the conventional group was 3.44±0.12 months (Figure 6(c)). In terms of the distribution of the number of stones, the proportion of patients with single stones in the AI group was 21.79% (17/78), the proportion of multiple stones was 78.21% (61/78); the proportions of patients with single stones and multiple stones in the conventional group accounted for 23.64% (26/110) and 76.36% (84/110), respectively (Figure 6(d)). After comparative analysis, the two groups were basically the same in gender distribution, age distribution, course of disease, and distribution of the number of stones, so there was no obvious statistical difference (P>0.05).Figure 6
Comparison on general data of patients in two group. Note: (a)–(d) illustrate the comparison on gender, age, course of disease, and the number of stones, respectively.
(a)(b)(c)(d)


### 3.3. Comparison on Observation Indicators for Patients in Two Groups

The statistics results of some of observation indicators for the two groups of patients are given in Table1, including operation time, postoperative drainage tube placement time, time required for recovery of gastrointestinal function, time required to get out of bed, and time spent in hospital. The observation and comparison revealed that the operation time (2.74±0.45 hours), postoperative drainage tube placement time (4.31±1.15 days), time required for recovery of gastrointestinal function (1.74±0.54 days), time required to get out of bed (1.14±0.55 hours), and time spent in hospital (9.94±1.45 days) for patients in the AI group were shorter than those in the conventional group, showing obvious statistical differences (P<0.05). Figure 7 shows the intraoperative blood loss and postoperative stone residual rate of the two groups of patients. Figure 7(a) shows the intraoperative blood loss of the two groups of patients. The intraoperative blood loss for patients in the AI group was 79.74±6.45mL, and that in the conventional group was 150.67±15.71mL. Figure 7(b) shows the residual status of stones in the two groups of patients. The number of patients with residual stones in the AI group was 0, and there were 12 patients with residual stones in the conventional group, accounting for 10.91%. After comparative analysis, it can be found that the intraoperative blood loss and postoperative stone residual rate of patients in the AI group were lower compared with those of the conventional group, and the differences were statistically notable (P<0.05).Table 1
Comparison on observation indicators for patients in the two groups.
Observation indicatorsAI group (n=78)Conventional group (n=110)Operation time (h)2.74±0.453.21±0.32Postoperative drainage tube placement time (d)4.31±1.1512.14±2.98Time required for recovery of gastrointestinal function (d)1.74±0.542.89±0.67Time required to get out of bed (h)1.14±0.552.09±0.87Time spent in hospital (d)9.94±1.4514.14±1.15Figure 7
Comparison on intraoperative blood loss and postoperative stone residual rate of patients. Note: (a) shows the comparison on intraoperative blood loss; (b) shows the comparison on postoperative stone residual rate.
(a)(b)


### 3.4. Comparison on Postoperative Complications

Based on the statistics of the results of this study, the postoperative complications of the two groups of patients were mostly biliary fistula, biliary hemorrhage, bile reflux esophagitis, acute pancreatitis, and incision infection (Figure8). In the AI group, the numbers of patients with biliary fistula, biliary hemorrhage, bile reflux esophagitis, acute pancreatitis, and incision infections were 0, 2, 1, 5, and 2, respectively. The total incidence of complications in the AI group was 10.26%. The numbers of patients with biliary fistula, biliary hemorrhage, bile reflux esophagitis, acute pancreatitis, and incision infections were 2, 4, 5, 9, and 6, respectively. After statistics and calculation analysis, the total incidence of complications in conventional group patients was 23.64%. Therefore, it was clear that the incidence of complications in the AI group was lower obviously than that in the conventional group, showing statistical difference (P<0.05).Figure 8
Comparison on postoperative complications. Note: A~F in the figure referred to biliary fistula, biliary hemorrhage, bile reflux esophagitis, acute pancreatitis, incision infection, and total incidence, respectively.


### 3.5. Comparison on Postoperative Gallbladder Function

Figure9 shows the comparison on average values of serum total bilirubin, direct bilirubin, alanine aminotransferase, aspartate aminotransferase, alkaline phosphatase, and glutamyl transpeptidase in the two groups of patients. After analysis, it was found that the serum total bilirubin (64.99±24.45μmol/L), direct bilirubin (93.79±22.11μmol/L), alanine aminotransferase (77.99±13.01U/L), aspartate aminotransferase (40.98±13.9U/L), alkaline phosphatase (211.77±101.91U/L), and glutamyl transpeptidase (423.12±109.21U/L) were lower remarkably than those in the conventional group, which were 30.13±33.25μmol/L, 166.11±122.09μmol/L, 116.89±100.99U/L, 89.08±13.01U/L, 328.09±110.01U/L, and 570.99±101.21U/L, respectively. The differences in above indicators between patients in the two groups were statistically great (P<0.05).Figure 9
Comparison on postoperative gallbladder function indicators. Note: A~F in the figure referred to serum total bilirubin, direct bilirubin, alanine aminotransferase, aspartate aminotransferase, alkaline phosphatase, and glutamyl transpeptidase, respectively.


## 3.1. Image Segmentation Results

Figure5 shows the captured images of the gallbladder stones and choledocholithiasis under laparoscopy and duodenoscope. Figures 5(a)–5(d) are the initial images under the endoscopy, and Figures 5(a1)–5(d1) are the images under the M-Unet segmentation. Figure 5(a) is a mirror image of the complete gallbladder of the abdominal cavity. It showed that the diameter of the gallbladder was slightly increased, but there was no inflammatory response on the surface. Figure 5(a1) shows the shape of the gallbladder segmented by M-Unet in the original image which performed. Figure 5(b) is an image of the bile duct after the surface mucosal tissue had been removed under laparoscopic surgery, and the shape of the bile duct can be clearly observed where the yellow arrow pointed. Figure 5(b1) shows the morphological performance of the bile duct after M-Unet segmentation. Figure 5(c) shows the state of stones in the bile duct observed under the duodenum meridian, as pointed by the yellow arrow. It can be concluded that the surface of the stones was smooth, and the overall colour was black. Compared with the diameter of the bile duct, the diameter of the stones was relatively small. Figure 5(c1) shows the morphological performance of bile duct stones obtained by M-Unet segmentation. Observation suggested that only the segmentation was the part shown by the plain film. Figure 5(d) shows a stone blocked in the bile duct orifice under a duodenoscope (pointed by the yellow arrow). It can be observed that the bile duct orifice had been cracked due to the blockage of the stone, and inflammation was visible. It also indirectly showed that the diameter of the stones in the figure was larger than the diameter of the bile duct, and the stones had to be crushed before taking them. Figure 5(d1) shows the stone part of the bile duct orifice segmented by M-Unet. After comparative analysis, the segmentation was relatively complete, and there was basically no missing segmentation. After calculation, the precision, recall, and the mean intersection ratio of the M-Unet-based segmentation algorithm were 94.56%, 96.56%, and 98.92%, respectively. The results were quite good and can be used in this study.Figure 5
The captured images of the gallbladder stones and choledocholithiasis under laparoscopy and duodenoscope. Note: (a) and (a1) show the images of the gallbladder; (b) and (b1) show the images of the bile duct; (c) and (d) show the images of the common bile duct; and (c1) and (d1) show the stone in the common bile duct. The yellow arrows on images (b), (c), and (d) referred to the cystic duct, stones in the common bile duct, and stones in the common bile duct, respectively.


## 3.2. Comparison on General Data of Patients

According to the research statistics, Figure6 shows the general treatment status of the two groups of patients in this study, including gender distribution, age distribution, average course of disease, and distribution of the number of stones. In terms of gender distribution, the proportion of men in the AI group was 48.72% (38/78) and the proportion of women was 51.28% (40/78); the proportions of men and women in the conventional group were 46.36% (51/110) and 53.64% (59/110), respectively (Figure 6(a)). In terms of age, the patients were divided into five stages: 25–35, 36–45, 46–55, 56–65, and >66 years old. The proportions of patients in five stages in the AI group were 8.97% (7/78), 19.23% (15/78), 29.49% (23/78), 15.38% (12/78), and 26.92% (21/78), respectively; while those in the conventional group were 7.27% (8/110), 21.82% (24/110), 24.55% (27/110), 18.18% (20/110), and 28.18% (31/110), respectively (Figure 6(b)). The average course of disease for patients in the AI group was 3.37±0.78 months, and that in the conventional group was 3.44±0.12 months (Figure 6(c)). In terms of the distribution of the number of stones, the proportion of patients with single stones in the AI group was 21.79% (17/78), the proportion of multiple stones was 78.21% (61/78); the proportions of patients with single stones and multiple stones in the conventional group accounted for 23.64% (26/110) and 76.36% (84/110), respectively (Figure 6(d)). After comparative analysis, the two groups were basically the same in gender distribution, age distribution, course of disease, and distribution of the number of stones, so there was no obvious statistical difference (P>0.05).Figure 6
Comparison on general data of patients in two group. Note: (a)–(d) illustrate the comparison on gender, age, course of disease, and the number of stones, respectively.
(a)(b)(c)(d)


## 3.3. Comparison on Observation Indicators for Patients in Two Groups

The statistics results of some of observation indicators for the two groups of patients are given in Table1, including operation time, postoperative drainage tube placement time, time required for recovery of gastrointestinal function, time required to get out of bed, and time spent in hospital. The observation and comparison revealed that the operation time (2.74±0.45 hours), postoperative drainage tube placement time (4.31±1.15 days), time required for recovery of gastrointestinal function (1.74±0.54 days), time required to get out of bed (1.14±0.55 hours), and time spent in hospital (9.94±1.45 days) for patients in the AI group were shorter than those in the conventional group, showing obvious statistical differences (P<0.05). Figure 7 shows the intraoperative blood loss and postoperative stone residual rate of the two groups of patients. Figure 7(a) shows the intraoperative blood loss of the two groups of patients. The intraoperative blood loss for patients in the AI group was 79.74±6.45mL, and that in the conventional group was 150.67±15.71mL. Figure 7(b) shows the residual status of stones in the two groups of patients. The number of patients with residual stones in the AI group was 0, and there were 12 patients with residual stones in the conventional group, accounting for 10.91%. After comparative analysis, it can be found that the intraoperative blood loss and postoperative stone residual rate of patients in the AI group were lower compared with those of the conventional group, and the differences were statistically notable (P<0.05).Table 1
Comparison on observation indicators for patients in the two groups.
Observation indicatorsAI group (n=78)Conventional group (n=110)Operation time (h)2.74±0.453.21±0.32Postoperative drainage tube placement time (d)4.31±1.1512.14±2.98Time required for recovery of gastrointestinal function (d)1.74±0.542.89±0.67Time required to get out of bed (h)1.14±0.552.09±0.87Time spent in hospital (d)9.94±1.4514.14±1.15Figure 7
Comparison on intraoperative blood loss and postoperative stone residual rate of patients. Note: (a) shows the comparison on intraoperative blood loss; (b) shows the comparison on postoperative stone residual rate.
(a)(b)


## 3.4. Comparison on Postoperative Complications

Based on the statistics of the results of this study, the postoperative complications of the two groups of patients were mostly biliary fistula, biliary hemorrhage, bile reflux esophagitis, acute pancreatitis, and incision infection (Figure8). In the AI group, the numbers of patients with biliary fistula, biliary hemorrhage, bile reflux esophagitis, acute pancreatitis, and incision infections were 0, 2, 1, 5, and 2, respectively. The total incidence of complications in the AI group was 10.26%. The numbers of patients with biliary fistula, biliary hemorrhage, bile reflux esophagitis, acute pancreatitis, and incision infections were 2, 4, 5, 9, and 6, respectively. After statistics and calculation analysis, the total incidence of complications in conventional group patients was 23.64%. Therefore, it was clear that the incidence of complications in the AI group was lower obviously than that in the conventional group, showing statistical difference (P<0.05).Figure 8
Comparison on postoperative complications. Note: A~F in the figure referred to biliary fistula, biliary hemorrhage, bile reflux esophagitis, acute pancreatitis, incision infection, and total incidence, respectively.


## 3.5. Comparison on Postoperative Gallbladder Function

Figure9 shows the comparison on average values of serum total bilirubin, direct bilirubin, alanine aminotransferase, aspartate aminotransferase, alkaline phosphatase, and glutamyl transpeptidase in the two groups of patients. After analysis, it was found that the serum total bilirubin (64.99±24.45μmol/L), direct bilirubin (93.79±22.11μmol/L), alanine aminotransferase (77.99±13.01U/L), aspartate aminotransferase (40.98±13.9U/L), alkaline phosphatase (211.77±101.91U/L), and glutamyl transpeptidase (423.12±109.21U/L) were lower remarkably than those in the conventional group, which were 30.13±33.25μmol/L, 166.11±122.09μmol/L, 116.89±100.99U/L, 89.08±13.01U/L, 328.09±110.01U/L, and 570.99±101.21U/L, respectively. The differences in above indicators between patients in the two groups were statistically great (P<0.05).Figure 9
Comparison on postoperative gallbladder function indicators. Note: A~F in the figure referred to serum total bilirubin, direct bilirubin, alanine aminotransferase, aspartate aminotransferase, alkaline phosphatase, and glutamyl transpeptidase, respectively.


## 4. Discussion

Cholelithiasis refers to the stone-like disease that occurs in the gallbladder and bile duct. It is not only a disease with a high incidence and a high incidence in China but also has a high incidence in the world. According to relevant statistical reports, about 10%~15% of European adult population suffer from gallstone disease, and the incidence of gallbladder stones combined with choledocholithiasis in patients with gallstone disease is 10%~20% [21, 22]. There are also related statistics suggesting that in the United States, there are about 2,000 new cases of gallstones appearing every year. Studies have shown that 18%~33% of symptomatic cholelithiasis is related to acute biliary pancreatitis [23]. The surgical treatment of biliary stones has undergone many improvements and is still being improved, mainly to provide patients with more efficient and convenient treatment methods. The surgical treatment of gallbladder stones has evolved from traditional open surgery to today’s minimally invasive surgical treatment under the guidance of endoscopy. Such series of changes have greatly reduced the pain of the patient during the treatment process and can further reduce the time required for the prognosis of the patients.In this study, the visual sensing technology based on the CNN system was applied to further optimize the surgical treatment of cholelithiasis guided by endoscopy and analyze its application effects through clinical experiments. Firstly, the improved CNN algorithm was adopted to optimize the visual sensing technology, which was tested and verified then. The results showed that the precision of the M-Unet-based segmentation algorithm reached 94.56%, the recall reached 96.56%, and the mean intersection ratio was 98.92%. Such results are basically consistent with the trend of a study on the multitarget detection and recognition system in the intelligent visual sensor network (in which the precision, accuracy, and mean intersection ratio based on the segmentation under intelligent algorithms were 96.63%, 56.92%, and 93.32%, respectively) [24]. A study on the application effects and functions of industrial visual perception technology in smart cities found that the improved CNN algorithm was superior to other algorithms in image and video processing [25]. Secondly, this technology was combined with endoscopic technology and applied to the surgical treatment of patients with gallbladder stones combined with choledocholithiasis. It was found in this study that in the AI group, the operation time (2.74±0.45h), postoperative drainage tube placement time (4.31±1.15d), time required for recovery of gastrointestinal function (1.74±0.54d), time required to get out of bed (1.14±0.55h), and time spent in hospital (9.94±1.45d) were all shorter compared with those in the conventional group, which were 3.21±0.32h, 12.14±2.98d, 2.89±0.67d, 2.09±0.87h, and 14.14±1.15h, showing statistical differences (P<0.05); the intraoperative blood loss (79.74±6.45mL) and residual status of stones (0%) in the AI group were much lower than those in the conventional group (P<0.05); in addition, the incidence of complications (10.26%) during the treatment was lower than that (23.64%) in the conventional group (P<0.05). What is more, the therapeutic effect of the biliary function index of the AI group was also much better than that of the conventional group. The above results are consistent with most other complication rates after treatment with laparoscopy combined with duodenoscope for treatment of gallbladder stones combined with common bile duct stones [4, 5, 26–28], providing support for laparoscopy combined with duodenoscope in the treatment of gallstones. Of course, AI-assisted visual sensing technology shows a wide range of applications, not just for assisted endoscopic treatment of gallbladder stones, which has been illustrated in a number of studies on the optimization of colonoscopy bowel preparation and the establishment and preliminary verification of the artificial intelligence-assisted colorectal polyp identification system [29–32]. AI-assisted visual sensing technology can also be used to prevent and control children’s myopia. A systematic study on the use of multipurpose 3D visual sensing technology to prevent and control children’s myopia illustrates this research [33]. AI-assisted visual sensing technology can also be applied to the identification and detection of drugs, which is specifically embodied in a research on intelligent and visual sensing technology and the application of this technology in the automatic visual detection of drugs [34]. Both the research results of this study and the related research results of predecessors reflect that it is natural and inevitable that intelligent and visual sensing technology is popularized in all aspects of people’s lives.


## 5. Conclusions

In this study, the visual sensing technology based on the CNN system was applied to further optimize the treatment of cholelithiasis guided by endoscopy and analyze its application effects through clinical experiments. The conclusions were summarized as follows. Firstly, CNN algorithm showed good effects in image segmentation processing. Secondly, based on the CNN algorithm, AI-assisted visual sensing technology can reasonably process the target image and has a good optimization effect. Thirdly, applying the visual sensing technology based on CNN algorithm to endoscopic technology for the treatment of gallbladder stones combined with choledocholithiasis can effectively improve the treatment effect. However, the representativeness of the research results was not up to standard due to the insufficient number of samples taken in this study, so further research was needed. This also gave certain empirical hints, which would be continuously summarized and corrected in subsequent research. However, it was proved in this study that visual sensing technology based on intelligent algorithms showed a good future in the medical field.



---

*Source: 5158577-2021-10-06.xml*