# Research on Basketball Goal Recognition Based on Image Processing and Improved Algorithm

Authors: Hangsheng Jiang
Journal: Security and Communication Networks
 (2021)

---

## Abstract

This paper studies the basketball goal recognition method based on image processing and improved algorithm to improve the accuracy of automatic recognition of basketball goal. The infrared spectrum image acquisition system is used to collect the basketball goal image. After the image is denoised by using the adaptive filtering algorithm, the wavelet analysis method is used to extract the features of basketball goal signal, which are input into the optimized deformable convolution neural network. Through the weighted sum of the values of each sampling point and the corresponding position authority of the block convolution core, the results are output as convolution operation. Combined with the depth feature of the same dimension, the full connection feature of the candidate target area is obtained to realize the basketball goal recognition. The experimental results show the following: the method can effectively identify basketball goals and the recognition error rate is low; the average accuracy of the automatic recognition results of basketball goals is as high as 98.4%; under the influence of different degrees of noise, the method is less affected by noise and has strong anti-interference ability.


---

## Body

## 1. Introduction

National Basketball Association (NBA) competition and China Basketball Association (CBA) competition are the most popular sport competitions in the current ball games. Through the understanding of NBA and CBA, it is found that they use the method of combining artificial and intelligent devices to achieve timing and scoring. NBA’s backboard is equipped with a camera. Every time a player is under the basket or making a layup, it will automatically take a picture. When the basketball falls from the basket and touches the net, the net has a dynamic response device, the rostrum receives the goal information, and the referee signals the rostrum at the same time to update the score and the timer. In January 22, 2014, in CBA, in a Beijing team vs. Foshan team game, there was a 24-second timer “stop,” once the game is interrupted, and finally it manually reads the watch. In such a professional competition, the appearance of such a scene is also worth thinking about [1]. NBA and CBA competitions are dominated by manpower, supplemented by equipment. At present, many places still use some artificial methods to score for basketball examinations, which will lead to an unfair phenomenon. At present, the commonly used shooting scoring methods are as follows: using artificial way to achieve basketball scoring, which is gradually replaced by some intelligent devices. Now many basketball games use infrared detection method to judge basketball goals [2]. Although the two intelligent detection methods of infrared detection and microswitch can effectively solve some problems existing in traditional artificial detection, they also have some shortcomings, such as easy to damage and high maintenance cost. So it is urgent to find a better device to replace the previous methods [3].With the rapid development of image processing technology, moving object detection technology in videos has been widely used. Especially in recent years, the application in sports, in which the video image processing technology, includes three parts of image acquisition, processing, and image secondary display [4, 5]. Similarly, with the rapid development of the times, more and more video processing applications are proposed. As a very important branch of vision technology [6], moving object detection technology has a strong scientific research value. Through research, it is found that the topological structure of the data communication network is an important factor for its security against attacks. In such networks, by changing the topology, the security robustness against intentional attack that aims at bringing down network nodes may vary. This bringing down is a kind of destruction and interruption threat that attacks on the availability of the network (i.e., attack on network resources and links). A large number of researchers at home and abroad have done a lot of research on it and achieved good research results. For example, Yuntao Cui proposed to extract the features of the target by using the Markov field after locating the target area; Eun Ryun et al. used the color components in the image to locate and recognize the target; Xue et al. proposed the key technology of moving target detection and recognition based on image sequence; Zhang et al. proposed the adaptive target recognition algorithm of image based on depth feature learning [7, 8]. These methods solve the problems of traditional image recognition algorithm, such as cumbersome process and difficult feature extraction, but in the actual application process, the comprehensive performance of basketball goal recognition is not high, which cannot avoid the problems of easy damage, high replacement rate, high installation and production cost, and misjudgment in the traditional fixed-point shooting device.The main contributions of this paper are as follows: this paper mainly studies the basketball goal recognition method based on image processing and improved algorithm, through the combination of image processing technology and improved convolution neural network to better solve some of the above problems.The rest of this paper is organized as follows: Section2 discusses basketball goal recognition based on image processing and improved algorithm and Section 3 shows simulation results analysis and discussion, and Section 4 concludes the paper with summary and future research directions.


## 2. Basketball Goal Recognition Based on Image Processing and Improved Algorithm

### 2.1. Acquisition of Basketball Goal Image

The main components of image acquisition system are image spectrometer, optical fiber halogen lamp, electronic control mobile platform, infrared camera, computer, and lens. The imaging principle of the system is that when the optical fiber halogen lamp irradiates and collects the basketball goal image, the basketball goal image collected in the instantaneous field of view is first imaged in the slit of the image spectrometer, and finally imaged in the charge coupled device (CCD) after passing through the grating and prism splitter assembly [9]. The spatial dimension is one dimension of CCD photosensitive plane parallel to the slit, and the spectral dimension is one dimension of CCD photosensitive plane perpendicular to the slit. In each row of photosensitive elements of spatial dimension, a spectral image of basketball goal is acquired, which means that each frame image in CCD corresponds to the multispectral image of basketball goal [10]. The spectral range of the system is 1000–1800 nm, the image resolution is 320 × 256 pixels, the frame rate is 100 Hz, the dynamic range is 68 dB, the exposure time is 1 μs–400 s, and the power is less than 5 W. The resolution of the image spectrometer is 5 nm, the slit width is 30 μm, the pixel size is 7.6 mm × 14.2 mm, the numerical aperture is F/2.0, and the light flux is more than 50% in the range of 1000–1800 nm.Before collecting the basketball goal image, the exposure time of the infrared camera should be determined to ensure the clarity of the basketball goal image; the speed of the electronic control mobile platform should be determined to ensure that the size and spatial resolution of the basketball goal image are not distorted. Through several experiments, it is proved that when the exposure time is 3 ms and the electronic control mobile platform is 0.59 mm/s, the basketball goal image is the clearest. Because the light source intensity is not evenly distributed in each wavelength, there is dark current noise in the infrared camera, resulting in a large noise in the collected basketball goal image in the wavelength with less light source intensity distribution, so it is necessary to calibrate the collected basketball goal image in black and white [11]. In the case of the system consistent with the acquisition of basketball goals, the standard white correction plate is scanned to obtain the all-white calibrated basketball goal image O, and the camera shutter is closed to implement image acquisition to obtain the all-black calibrated basketball goal image B. Finally, the absolute basketball goal image I is converted into the relative basketball goal image S. The equation is as follows:(1)S=I−BO−B.


### 2.2. Preprocessing of Basketball Goal Image

Att=0, 30 evenly distributed particles in the 50∗50 neighborhood around the target basketball goal image are randomly selected.LetSt−1E and St−1i be the collection result and particle set at time t−1, and π˜t−1i and ct−1i be the weight and cumulative probability of each particle, i=1, …, nt−1. The number of particles is nt−1, and the template of basketball goal image at time t is qujt, j=1,2.The adaptive particle filter algorithm is used to process basketball goal image, and the steps are as follows:Step 1.
resample according to the weight of each particle to getSt−1′i, i=1,…,nt−1:(1)
Form a random number with uniform distribution,r∈0,1.(2)
Calculate the minimumJ of ct−1j≥r and obtain St−1′i=St−1j.Step 2.
Calculate the particle setSti at time t.
The motion model of Gaussian noise varianceσ2 selected by adaptive particle filter algorithm without occlusion is as follows:(2)Sti=St−1′i−k×nt−1+St−1E−St−2E+wt−1i,wt−1i∼N0,σ2,i=1,…,nt.
In equation (10), k=i−nt−1/nt−1 denotes the large direction, wt−1i is the binary Gaussian random noise, and wt−1i∼N0,σ2 is the range of Brownian motion of the particle.
Under occlusion, there is only Gaussian noise in the motion model, and the fixed values ofσ and nt are 5 and 20. The particle state after Brownian motion is obtained as follows:(3)S‵ti=St−i′i+wt−1i,wt−1i∼N0,25,i=1,…,20,where S‵ti is further iterated by the mean shift algorithm to obtain the optimized particle Sti.Step 3.
Calculate the normalized weightπ˜ti according to the matching degree of each particle histogram pujSii and template qujt:(4)πii=exp−dSti22σ02.
In equation (12),(5)dSti=1ρSti=1−∑u1=1Ncr×Ncbpu1Stiqu1t×∑u2=1Nypu2Stiqu2t,π˜ti=πti∑j=1ntπij,i=1,…,nt,where σ0 is an important parameter. When the value of σ0 is selected, the weight difference of each particle should be kept in an appropriate range, and the effectiveness and diversity of particles during resampling should be ensured [12–14]. Ncr × Ncb and Nv are the two-dimensional histogram and one-dimensional histogram in the target space of basketball goal image, respectively, and ρ is the histogram matching degree.Step 4.
Calculate the cumulative probabilitycti of particles(6)ct0=0,cti=cti−1+π˜ti,i=1,…,nt.Step 5.
Calculate the weighted average stateStE.(7)StE=∑i=1ntπ˜tiSti.


### 2.3. Feature Extraction of Basketball Goal Image

After preprocessing the basketball goal image, wavelet analysis is used to extract the image features of basketball goal. Suppose that the equation of binary discrete wavelet function is as follows:(8)ψy,kt=2−y/2ψ2yt−k,where the number of frames between frames is t, the number of decomposition layers is k, and the number of image frames is y.The discrete wavelet transform equation of corresponding functionf(t) is as follows:(9)Pfy,k=ψy,kt,ft=∫Kftψ¯y,ktdt,where the mean value of differential cumulative image is P, the difference between frames is d, and the energy level of time-domain waveform is K.If the discrete wavelet functionψy,kt is formed into orthonormal basis L2K, then ft∈L2K can be linearly combined with orthonormal basis, and the equation is as follows:(10)ft=∑y,kPfy,kψy,kt.In the case of orthogonal basis, the discrete wavelet is replaced by the mapping which ensures the invariance of norm, and there is no information redundancy. Then the equation is as follows:(11)ft,ft=∫−∞−∞ft2dt=∑y,kPfy,k2.After replacing the discrete wavelet, the original signal is decomposed into each frequency band. Let the sampling frequency of the signal befs, so in the k-th decomposition, the frequency band range of the approaching signal is (0, fs/2k+1), and the frequency band range of the detail signal is (fs/2k+1, fs/2k). The second power of the coefficients is the same as the total energy of the waveform in the same time domain [15, 16]. The equation of basketball goal signal extraction based on energy feature is as follows:(12)U=12−KN−1∑y=1Pfy,k2.The number of frame difference isN.


### 2.4. Basketball Goal Image Recognition

#### 2.4.1. Convolutional Neural Network

The single neuron model of convolutional neural network is shown in Figure1.Figure 1
Single neuron model.The input signalxi of convolutional neural network neurons comes from the output of feedforward n neurons. After multiplying the input signal xi by the weight wi, the total input value is obtained by adding it to the offset b. Compared with the total input received by the neuron and the set threshold θ, the “activation function” is used to realize the activation processing, and the neuron output is obtained [17]. A simple 4-layer convolutional neural network model is shown in Figure 2.Figure 2
Neural network model.Convolutional neural network is constructed by connecting a large number of neurons of the same form according to a certain organizational hierarchy. Each column is called the layer of neural network, which is mainly divided into input layer, hidden layer, and output layer. The input of each layer is the output of the previous layer, and the output is the input of the latter layer [18].


#### 2.4.2. Basketball Goal Recognition Based on Improved Convolution Neural Network

In order to realize the accurate recognition of basketball goals, convolution neural network is optimized, and the recognition of basketball goals is realized through the regular block convolution operation. The training part of optimized convolutional neural network mainly includes prediction subnetwork and identification subnetwork of target region, which share convolutional neural network structure [19].Based on the characteristics of the basketball goal image obtained above, the characteristic image of the basketball goal image is obtained. The weighted sum of the values of each sampling point and the corresponding position authority of the block convolution kernel is performed, and the results are output as convolution operation. The receptive field of square convolution kernel is selected as regular region.The definition of any pointq0 in the output feature map y of convolution operation is given as follows:(13)yq0=∑qn∈Rwqn⋅xq0+qn,where the elements in the receptive field area are expressed as qn, the feature map of basketball goal image is expressed as x, and the convolution operation is expressed as w.The offsetVqn|n=1,2,…,N is added into the deformation convolution operation, and the number of elements in the receptive field is described as N. Formula (13) is transformed into the following:(14)yq0=∑qn∈Rwqn⋅xq0+qn+Vqn.By using the convolution kernel convolution operation of the offset variable, the basketball goal image feature map is input and the offset feature map is output, and the total number of feature values of the offset value of the deformation convolution operation is obtained, which can be reflected in each position in the map, and the final feature map is obtained by the offset value.The convolution kernel with the size of sliding window is selected at the center pixel position of each sliding window to collect the depth featureXi. The candidate detection frame is described by Di=dix,diy,diw,dih, the x-axis coordinate of the upper left corner of the candidate detection frame is described by dix, the y-axis coordinate of the upper left corner of the candidate detection frame is described by diy, the width of the candidate detection frame is described by diw, and the height of the candidate detection frame is described by dih.If the candidate detection frame is a positive sample, the overlap ratio of truth value frameDi∗ and candidate rectangle frame is higher than 0.3, which is described by Yi≥1; if the candidate detection frame is a negative sample, the overlap ratio of truth value frame Di∗ and candidate rectangle frame is lower than 0.3, which is described by Yi=0. The equation of overlapping ratio is as follows:(15)a=areaDi∩Di∗areaDi∪Di∗.The intersection area of different rectangles is described byareaDi∩Di∗, and the union area of different rectangles is described by areaDi∪Di∗.Sm=Xi,Yi,Dii=1N is used to describe the training sample set, and the candidate target regions are predicted by sliding windows of different sizes. The loss function formula of regression and classification is obtained as(16)lmX,Y,D|W=LeqX,Y+λY≥1LbD∗,D^,where the cross entropy loss function of model parameters is described by W; the cross entropy loss function of measuring classification loss is described by LeqX,Y; the probability of various targets is expressed by qX, and the balance parameter is expressed by λ; the detection box of background not participating in regression operation is described by Y≥1, and the detection box of completing regression operation is described by D^; the Lb-type loss function is described by Lb, that is,(17)LbD∗,D^=fL1D∗−D^,where(18)fL1x=0.5x2,ifx<1,x−0.5,otherwise.The overall loss function equation of candidate target area prediction subnetwork is obtained as follows:(19)LOW=∑m=19∑i∈SmμmlmXi,Yi,Di|W,where μm is described as a weighted parameter.The gradient descent method is used to solve the prediction subnetwork of candidate target region, and the prediction result is realized by the subnetwork of target region identification of convolution neural network.Through the pooling layer of convolution neural network, the candidate target regions are pooled to obtain the depth features of the same dimension. The depth features are input to the full connection layer to obtain the full connection features of the candidate target regions and realize the final position regression and classification of the candidate target regions [20]. The multi-directional rotation of the basketball goal image will cause the pooling operation in the candidate target region to be mixed with the background region, so it is necessary to introduce the offset variable into the region pooling operation through the deformation convolution method.Let the volume of the rectangular region bew×h, and the size of the rectangular region is transformed into k×k by pooling operation. The conversion formula is as follows:(20)yi,j=∑p∈bini,jxq0+qnij,where the number of pixels in each subregion is described as nij and the upper left corner coordinate is described as q0. By adding the offset Vqij to equation (20), it is concluded that(21)yi,j=∑q∈bini,jxq0+q+Δqijnij,where 0≤i, j<k.According to the above analysis, the depth features of each candidate target region are extracted in the residual block group by using the subnetwork of target region identification of deformation pooling operation, and the loss function can be transformed into(22)LW,Wd=∑m=19∑i∈SmμmlmXi,Yi,Di|W+∑i∈S10μ10l10Xi,Yi,Di|W,Wd,where the weighted parameter of the target area discrimination subnetwork is expressed as μ10, the training set of the subnetwork of target area discrimination is expressed as S10, and the depth feature of the target area is expressed as Wd.The process of basketball goal recognition based on the improved convolution neural network is as follows: the subnetwork of target region prediction is used to initialize the subnetwork of target region identification and the added full connection layer; the subnetwork is trained jointly and the learning rate is set; after the training, the basketball goal image to be recognized is input; and the target with the highest confidence result is taken as the target detection result. The detection frame with high overlap ratio can get the result of basketball goal through nonmaximum suppression.In the last three years, many methods are proposed to handle the problem of basketball goal automation prediction; here we introduced three outstanding methods such as GNB [21], RNB [22], and CNB [23], which can be used to solve the related works taking different kinds of network structures. GNB is a graph-based network that builds connections between different risk nodes. And RNB uses a specific loss structure to keep the similarity of real and predicted goals differences. CNB is the basic model that needs more computation consuming more space to obtain the desiring performances. However, these methods have their disadvantages, respectively. GNB is too slow, RNB is so complicated, and CNB also needs more spaces.In this paper, we utilize the entropy loss function to build the model for our research problems. It can be defined as follows:lossx,y=∑i=1n−pilog1−pi, where x and y are represented as the real basketball goals’ score and y means the predicted score and difficult of our proposal. Pi means the probability of them when they are similar. The bigger the value of the loss, the worse our proposal performed. And our proposal is used to train a model that fits the real and predicted basketball goals, so that the machine can assist the actions predicted. And compared with the three methods, our proposal can deal with the problems easily, and we also need a smaller computation space to build our model. However, our model may obtain a relatively lower accuracy than others sometime which may lead the prediction to be unstable. The other methods are recently proposed methods, and they have their own advantages introduced as before and, however, also have their disadvantages, respectively. Here, our proposal can integrate their advantages well and avoid their shortcomings at the same time, so we compared each other. The prediction result of target recognition authenticity is shown in Table 1.Table 1
The prediction result of target recognition authenticity.
Real predictionPositiveNegativePositiveTPFNNegativeFPTN


## 2.1. Acquisition of Basketball Goal Image

The main components of image acquisition system are image spectrometer, optical fiber halogen lamp, electronic control mobile platform, infrared camera, computer, and lens. The imaging principle of the system is that when the optical fiber halogen lamp irradiates and collects the basketball goal image, the basketball goal image collected in the instantaneous field of view is first imaged in the slit of the image spectrometer, and finally imaged in the charge coupled device (CCD) after passing through the grating and prism splitter assembly [9]. The spatial dimension is one dimension of CCD photosensitive plane parallel to the slit, and the spectral dimension is one dimension of CCD photosensitive plane perpendicular to the slit. In each row of photosensitive elements of spatial dimension, a spectral image of basketball goal is acquired, which means that each frame image in CCD corresponds to the multispectral image of basketball goal [10]. The spectral range of the system is 1000–1800 nm, the image resolution is 320 × 256 pixels, the frame rate is 100 Hz, the dynamic range is 68 dB, the exposure time is 1 μs–400 s, and the power is less than 5 W. The resolution of the image spectrometer is 5 nm, the slit width is 30 μm, the pixel size is 7.6 mm × 14.2 mm, the numerical aperture is F/2.0, and the light flux is more than 50% in the range of 1000–1800 nm.Before collecting the basketball goal image, the exposure time of the infrared camera should be determined to ensure the clarity of the basketball goal image; the speed of the electronic control mobile platform should be determined to ensure that the size and spatial resolution of the basketball goal image are not distorted. Through several experiments, it is proved that when the exposure time is 3 ms and the electronic control mobile platform is 0.59 mm/s, the basketball goal image is the clearest. Because the light source intensity is not evenly distributed in each wavelength, there is dark current noise in the infrared camera, resulting in a large noise in the collected basketball goal image in the wavelength with less light source intensity distribution, so it is necessary to calibrate the collected basketball goal image in black and white [11]. In the case of the system consistent with the acquisition of basketball goals, the standard white correction plate is scanned to obtain the all-white calibrated basketball goal image O, and the camera shutter is closed to implement image acquisition to obtain the all-black calibrated basketball goal image B. Finally, the absolute basketball goal image I is converted into the relative basketball goal image S. The equation is as follows:(1)S=I−BO−B.


## 2.2. Preprocessing of Basketball Goal Image

Att=0, 30 evenly distributed particles in the 50∗50 neighborhood around the target basketball goal image are randomly selected.LetSt−1E and St−1i be the collection result and particle set at time t−1, and π˜t−1i and ct−1i be the weight and cumulative probability of each particle, i=1, …, nt−1. The number of particles is nt−1, and the template of basketball goal image at time t is qujt, j=1,2.The adaptive particle filter algorithm is used to process basketball goal image, and the steps are as follows:Step 1.
resample according to the weight of each particle to getSt−1′i, i=1,…,nt−1:(1)
Form a random number with uniform distribution,r∈0,1.(2)
Calculate the minimumJ of ct−1j≥r and obtain St−1′i=St−1j.Step 2.
Calculate the particle setSti at time t.
The motion model of Gaussian noise varianceσ2 selected by adaptive particle filter algorithm without occlusion is as follows:(2)Sti=St−1′i−k×nt−1+St−1E−St−2E+wt−1i,wt−1i∼N0,σ2,i=1,…,nt.
In equation (10), k=i−nt−1/nt−1 denotes the large direction, wt−1i is the binary Gaussian random noise, and wt−1i∼N0,σ2 is the range of Brownian motion of the particle.
Under occlusion, there is only Gaussian noise in the motion model, and the fixed values ofσ and nt are 5 and 20. The particle state after Brownian motion is obtained as follows:(3)S‵ti=St−i′i+wt−1i,wt−1i∼N0,25,i=1,…,20,where S‵ti is further iterated by the mean shift algorithm to obtain the optimized particle Sti.Step 3.
Calculate the normalized weightπ˜ti according to the matching degree of each particle histogram pujSii and template qujt:(4)πii=exp−dSti22σ02.
In equation (12),(5)dSti=1ρSti=1−∑u1=1Ncr×Ncbpu1Stiqu1t×∑u2=1Nypu2Stiqu2t,π˜ti=πti∑j=1ntπij,i=1,…,nt,where σ0 is an important parameter. When the value of σ0 is selected, the weight difference of each particle should be kept in an appropriate range, and the effectiveness and diversity of particles during resampling should be ensured [12–14]. Ncr × Ncb and Nv are the two-dimensional histogram and one-dimensional histogram in the target space of basketball goal image, respectively, and ρ is the histogram matching degree.Step 4.
Calculate the cumulative probabilitycti of particles(6)ct0=0,cti=cti−1+π˜ti,i=1,…,nt.Step 5.
Calculate the weighted average stateStE.(7)StE=∑i=1ntπ˜tiSti.


## 2.3. Feature Extraction of Basketball Goal Image

After preprocessing the basketball goal image, wavelet analysis is used to extract the image features of basketball goal. Suppose that the equation of binary discrete wavelet function is as follows:(8)ψy,kt=2−y/2ψ2yt−k,where the number of frames between frames is t, the number of decomposition layers is k, and the number of image frames is y.The discrete wavelet transform equation of corresponding functionf(t) is as follows:(9)Pfy,k=ψy,kt,ft=∫Kftψ¯y,ktdt,where the mean value of differential cumulative image is P, the difference between frames is d, and the energy level of time-domain waveform is K.If the discrete wavelet functionψy,kt is formed into orthonormal basis L2K, then ft∈L2K can be linearly combined with orthonormal basis, and the equation is as follows:(10)ft=∑y,kPfy,kψy,kt.In the case of orthogonal basis, the discrete wavelet is replaced by the mapping which ensures the invariance of norm, and there is no information redundancy. Then the equation is as follows:(11)ft,ft=∫−∞−∞ft2dt=∑y,kPfy,k2.After replacing the discrete wavelet, the original signal is decomposed into each frequency band. Let the sampling frequency of the signal befs, so in the k-th decomposition, the frequency band range of the approaching signal is (0, fs/2k+1), and the frequency band range of the detail signal is (fs/2k+1, fs/2k). The second power of the coefficients is the same as the total energy of the waveform in the same time domain [15, 16]. The equation of basketball goal signal extraction based on energy feature is as follows:(12)U=12−KN−1∑y=1Pfy,k2.The number of frame difference isN.


## 2.4. Basketball Goal Image Recognition

### 2.4.1. Convolutional Neural Network

The single neuron model of convolutional neural network is shown in Figure1.Figure 1
Single neuron model.The input signalxi of convolutional neural network neurons comes from the output of feedforward n neurons. After multiplying the input signal xi by the weight wi, the total input value is obtained by adding it to the offset b. Compared with the total input received by the neuron and the set threshold θ, the “activation function” is used to realize the activation processing, and the neuron output is obtained [17]. A simple 4-layer convolutional neural network model is shown in Figure 2.Figure 2
Neural network model.Convolutional neural network is constructed by connecting a large number of neurons of the same form according to a certain organizational hierarchy. Each column is called the layer of neural network, which is mainly divided into input layer, hidden layer, and output layer. The input of each layer is the output of the previous layer, and the output is the input of the latter layer [18].


### 2.4.2. Basketball Goal Recognition Based on Improved Convolution Neural Network

In order to realize the accurate recognition of basketball goals, convolution neural network is optimized, and the recognition of basketball goals is realized through the regular block convolution operation. The training part of optimized convolutional neural network mainly includes prediction subnetwork and identification subnetwork of target region, which share convolutional neural network structure [19].Based on the characteristics of the basketball goal image obtained above, the characteristic image of the basketball goal image is obtained. The weighted sum of the values of each sampling point and the corresponding position authority of the block convolution kernel is performed, and the results are output as convolution operation. The receptive field of square convolution kernel is selected as regular region.The definition of any pointq0 in the output feature map y of convolution operation is given as follows:(13)yq0=∑qn∈Rwqn⋅xq0+qn,where the elements in the receptive field area are expressed as qn, the feature map of basketball goal image is expressed as x, and the convolution operation is expressed as w.The offsetVqn|n=1,2,…,N is added into the deformation convolution operation, and the number of elements in the receptive field is described as N. Formula (13) is transformed into the following:(14)yq0=∑qn∈Rwqn⋅xq0+qn+Vqn.By using the convolution kernel convolution operation of the offset variable, the basketball goal image feature map is input and the offset feature map is output, and the total number of feature values of the offset value of the deformation convolution operation is obtained, which can be reflected in each position in the map, and the final feature map is obtained by the offset value.The convolution kernel with the size of sliding window is selected at the center pixel position of each sliding window to collect the depth featureXi. The candidate detection frame is described by Di=dix,diy,diw,dih, the x-axis coordinate of the upper left corner of the candidate detection frame is described by dix, the y-axis coordinate of the upper left corner of the candidate detection frame is described by diy, the width of the candidate detection frame is described by diw, and the height of the candidate detection frame is described by dih.If the candidate detection frame is a positive sample, the overlap ratio of truth value frameDi∗ and candidate rectangle frame is higher than 0.3, which is described by Yi≥1; if the candidate detection frame is a negative sample, the overlap ratio of truth value frame Di∗ and candidate rectangle frame is lower than 0.3, which is described by Yi=0. The equation of overlapping ratio is as follows:(15)a=areaDi∩Di∗areaDi∪Di∗.The intersection area of different rectangles is described byareaDi∩Di∗, and the union area of different rectangles is described by areaDi∪Di∗.Sm=Xi,Yi,Dii=1N is used to describe the training sample set, and the candidate target regions are predicted by sliding windows of different sizes. The loss function formula of regression and classification is obtained as(16)lmX,Y,D|W=LeqX,Y+λY≥1LbD∗,D^,where the cross entropy loss function of model parameters is described by W; the cross entropy loss function of measuring classification loss is described by LeqX,Y; the probability of various targets is expressed by qX, and the balance parameter is expressed by λ; the detection box of background not participating in regression operation is described by Y≥1, and the detection box of completing regression operation is described by D^; the Lb-type loss function is described by Lb, that is,(17)LbD∗,D^=fL1D∗−D^,where(18)fL1x=0.5x2,ifx<1,x−0.5,otherwise.The overall loss function equation of candidate target area prediction subnetwork is obtained as follows:(19)LOW=∑m=19∑i∈SmμmlmXi,Yi,Di|W,where μm is described as a weighted parameter.The gradient descent method is used to solve the prediction subnetwork of candidate target region, and the prediction result is realized by the subnetwork of target region identification of convolution neural network.Through the pooling layer of convolution neural network, the candidate target regions are pooled to obtain the depth features of the same dimension. The depth features are input to the full connection layer to obtain the full connection features of the candidate target regions and realize the final position regression and classification of the candidate target regions [20]. The multi-directional rotation of the basketball goal image will cause the pooling operation in the candidate target region to be mixed with the background region, so it is necessary to introduce the offset variable into the region pooling operation through the deformation convolution method.Let the volume of the rectangular region bew×h, and the size of the rectangular region is transformed into k×k by pooling operation. The conversion formula is as follows:(20)yi,j=∑p∈bini,jxq0+qnij,where the number of pixels in each subregion is described as nij and the upper left corner coordinate is described as q0. By adding the offset Vqij to equation (20), it is concluded that(21)yi,j=∑q∈bini,jxq0+q+Δqijnij,where 0≤i, j<k.According to the above analysis, the depth features of each candidate target region are extracted in the residual block group by using the subnetwork of target region identification of deformation pooling operation, and the loss function can be transformed into(22)LW,Wd=∑m=19∑i∈SmμmlmXi,Yi,Di|W+∑i∈S10μ10l10Xi,Yi,Di|W,Wd,where the weighted parameter of the target area discrimination subnetwork is expressed as μ10, the training set of the subnetwork of target area discrimination is expressed as S10, and the depth feature of the target area is expressed as Wd.The process of basketball goal recognition based on the improved convolution neural network is as follows: the subnetwork of target region prediction is used to initialize the subnetwork of target region identification and the added full connection layer; the subnetwork is trained jointly and the learning rate is set; after the training, the basketball goal image to be recognized is input; and the target with the highest confidence result is taken as the target detection result. The detection frame with high overlap ratio can get the result of basketball goal through nonmaximum suppression.In the last three years, many methods are proposed to handle the problem of basketball goal automation prediction; here we introduced three outstanding methods such as GNB [21], RNB [22], and CNB [23], which can be used to solve the related works taking different kinds of network structures. GNB is a graph-based network that builds connections between different risk nodes. And RNB uses a specific loss structure to keep the similarity of real and predicted goals differences. CNB is the basic model that needs more computation consuming more space to obtain the desiring performances. However, these methods have their disadvantages, respectively. GNB is too slow, RNB is so complicated, and CNB also needs more spaces.In this paper, we utilize the entropy loss function to build the model for our research problems. It can be defined as follows:lossx,y=∑i=1n−pilog1−pi, where x and y are represented as the real basketball goals’ score and y means the predicted score and difficult of our proposal. Pi means the probability of them when they are similar. The bigger the value of the loss, the worse our proposal performed. And our proposal is used to train a model that fits the real and predicted basketball goals, so that the machine can assist the actions predicted. And compared with the three methods, our proposal can deal with the problems easily, and we also need a smaller computation space to build our model. However, our model may obtain a relatively lower accuracy than others sometime which may lead the prediction to be unstable. The other methods are recently proposed methods, and they have their own advantages introduced as before and, however, also have their disadvantages, respectively. Here, our proposal can integrate their advantages well and avoid their shortcomings at the same time, so we compared each other. The prediction result of target recognition authenticity is shown in Table 1.Table 1
The prediction result of target recognition authenticity.
Real predictionPositiveNegativePositiveTPFNNegativeFPTN


## 2.4.1. Convolutional Neural Network

The single neuron model of convolutional neural network is shown in Figure1.Figure 1
Single neuron model.The input signalxi of convolutional neural network neurons comes from the output of feedforward n neurons. After multiplying the input signal xi by the weight wi, the total input value is obtained by adding it to the offset b. Compared with the total input received by the neuron and the set threshold θ, the “activation function” is used to realize the activation processing, and the neuron output is obtained [17]. A simple 4-layer convolutional neural network model is shown in Figure 2.Figure 2
Neural network model.Convolutional neural network is constructed by connecting a large number of neurons of the same form according to a certain organizational hierarchy. Each column is called the layer of neural network, which is mainly divided into input layer, hidden layer, and output layer. The input of each layer is the output of the previous layer, and the output is the input of the latter layer [18].


## 2.4.2. Basketball Goal Recognition Based on Improved Convolution Neural Network

In order to realize the accurate recognition of basketball goals, convolution neural network is optimized, and the recognition of basketball goals is realized through the regular block convolution operation. The training part of optimized convolutional neural network mainly includes prediction subnetwork and identification subnetwork of target region, which share convolutional neural network structure [19].Based on the characteristics of the basketball goal image obtained above, the characteristic image of the basketball goal image is obtained. The weighted sum of the values of each sampling point and the corresponding position authority of the block convolution kernel is performed, and the results are output as convolution operation. The receptive field of square convolution kernel is selected as regular region.The definition of any pointq0 in the output feature map y of convolution operation is given as follows:(13)yq0=∑qn∈Rwqn⋅xq0+qn,where the elements in the receptive field area are expressed as qn, the feature map of basketball goal image is expressed as x, and the convolution operation is expressed as w.The offsetVqn|n=1,2,…,N is added into the deformation convolution operation, and the number of elements in the receptive field is described as N. Formula (13) is transformed into the following:(14)yq0=∑qn∈Rwqn⋅xq0+qn+Vqn.By using the convolution kernel convolution operation of the offset variable, the basketball goal image feature map is input and the offset feature map is output, and the total number of feature values of the offset value of the deformation convolution operation is obtained, which can be reflected in each position in the map, and the final feature map is obtained by the offset value.The convolution kernel with the size of sliding window is selected at the center pixel position of each sliding window to collect the depth featureXi. The candidate detection frame is described by Di=dix,diy,diw,dih, the x-axis coordinate of the upper left corner of the candidate detection frame is described by dix, the y-axis coordinate of the upper left corner of the candidate detection frame is described by diy, the width of the candidate detection frame is described by diw, and the height of the candidate detection frame is described by dih.If the candidate detection frame is a positive sample, the overlap ratio of truth value frameDi∗ and candidate rectangle frame is higher than 0.3, which is described by Yi≥1; if the candidate detection frame is a negative sample, the overlap ratio of truth value frame Di∗ and candidate rectangle frame is lower than 0.3, which is described by Yi=0. The equation of overlapping ratio is as follows:(15)a=areaDi∩Di∗areaDi∪Di∗.The intersection area of different rectangles is described byareaDi∩Di∗, and the union area of different rectangles is described by areaDi∪Di∗.Sm=Xi,Yi,Dii=1N is used to describe the training sample set, and the candidate target regions are predicted by sliding windows of different sizes. The loss function formula of regression and classification is obtained as(16)lmX,Y,D|W=LeqX,Y+λY≥1LbD∗,D^,where the cross entropy loss function of model parameters is described by W; the cross entropy loss function of measuring classification loss is described by LeqX,Y; the probability of various targets is expressed by qX, and the balance parameter is expressed by λ; the detection box of background not participating in regression operation is described by Y≥1, and the detection box of completing regression operation is described by D^; the Lb-type loss function is described by Lb, that is,(17)LbD∗,D^=fL1D∗−D^,where(18)fL1x=0.5x2,ifx<1,x−0.5,otherwise.The overall loss function equation of candidate target area prediction subnetwork is obtained as follows:(19)LOW=∑m=19∑i∈SmμmlmXi,Yi,Di|W,where μm is described as a weighted parameter.The gradient descent method is used to solve the prediction subnetwork of candidate target region, and the prediction result is realized by the subnetwork of target region identification of convolution neural network.Through the pooling layer of convolution neural network, the candidate target regions are pooled to obtain the depth features of the same dimension. The depth features are input to the full connection layer to obtain the full connection features of the candidate target regions and realize the final position regression and classification of the candidate target regions [20]. The multi-directional rotation of the basketball goal image will cause the pooling operation in the candidate target region to be mixed with the background region, so it is necessary to introduce the offset variable into the region pooling operation through the deformation convolution method.Let the volume of the rectangular region bew×h, and the size of the rectangular region is transformed into k×k by pooling operation. The conversion formula is as follows:(20)yi,j=∑p∈bini,jxq0+qnij,where the number of pixels in each subregion is described as nij and the upper left corner coordinate is described as q0. By adding the offset Vqij to equation (20), it is concluded that(21)yi,j=∑q∈bini,jxq0+q+Δqijnij,where 0≤i, j<k.According to the above analysis, the depth features of each candidate target region are extracted in the residual block group by using the subnetwork of target region identification of deformation pooling operation, and the loss function can be transformed into(22)LW,Wd=∑m=19∑i∈SmμmlmXi,Yi,Di|W+∑i∈S10μ10l10Xi,Yi,Di|W,Wd,where the weighted parameter of the target area discrimination subnetwork is expressed as μ10, the training set of the subnetwork of target area discrimination is expressed as S10, and the depth feature of the target area is expressed as Wd.The process of basketball goal recognition based on the improved convolution neural network is as follows: the subnetwork of target region prediction is used to initialize the subnetwork of target region identification and the added full connection layer; the subnetwork is trained jointly and the learning rate is set; after the training, the basketball goal image to be recognized is input; and the target with the highest confidence result is taken as the target detection result. The detection frame with high overlap ratio can get the result of basketball goal through nonmaximum suppression.In the last three years, many methods are proposed to handle the problem of basketball goal automation prediction; here we introduced three outstanding methods such as GNB [21], RNB [22], and CNB [23], which can be used to solve the related works taking different kinds of network structures. GNB is a graph-based network that builds connections between different risk nodes. And RNB uses a specific loss structure to keep the similarity of real and predicted goals differences. CNB is the basic model that needs more computation consuming more space to obtain the desiring performances. However, these methods have their disadvantages, respectively. GNB is too slow, RNB is so complicated, and CNB also needs more spaces.In this paper, we utilize the entropy loss function to build the model for our research problems. It can be defined as follows:lossx,y=∑i=1n−pilog1−pi, where x and y are represented as the real basketball goals’ score and y means the predicted score and difficult of our proposal. Pi means the probability of them when they are similar. The bigger the value of the loss, the worse our proposal performed. And our proposal is used to train a model that fits the real and predicted basketball goals, so that the machine can assist the actions predicted. And compared with the three methods, our proposal can deal with the problems easily, and we also need a smaller computation space to build our model. However, our model may obtain a relatively lower accuracy than others sometime which may lead the prediction to be unstable. The other methods are recently proposed methods, and they have their own advantages introduced as before and, however, also have their disadvantages, respectively. Here, our proposal can integrate their advantages well and avoid their shortcomings at the same time, so we compared each other. The prediction result of target recognition authenticity is shown in Table 1.Table 1
The prediction result of target recognition authenticity.
Real predictionPositiveNegativePositiveTPFNNegativeFPTN


## 3. Experimental Analysis

Taking the basketball goals of a basketball game in a gymnasium as an example, the number of basketball goals of the game is 30 by manual counting. The method is used to collect and identify the basketball goals of the game and test the feasibility and accuracy of the automatic recognition of basketball goals. We use the confusion matrix, accuracy, precision, andF1-score to evaluate the model performance. The matrix can be defined as follows:(23)accuracy=TP+TNTP+TN+FP+FN,F1−score=2∗precise∗recallprecise+recall,(24)precise=TPTP+FP,recall=TPTP+FN.Convolution neural network is trained by using the training sample set, and then convolution neural network is tested by using the test sample set. The convolution neural network is optimized by regular block convolution to obtain the best structural parameters of adaptive convolution neural network. Table2 shows the structural parameters of adaptive convolution neural network with different number of input nodes.Table 2
Structural parameters of adaptive convolutional neural network.
Number of input nodes (A)Maximum number of training steps (step)Network training accuracy1270.0012280.0013260.0014200.0015150.0016330.0017250.0018220.0019480.00110550.001It can be seen from Table2 that the number of input nodes does not affect the network training accuracy, and the maximum number of training steps is the least when the number of input nodes is 5. Based on this, in the follow-up experiment process, the adaptive convolution neural network selects 5 input nodes, and the output result is the final experimental data, which saves the time of automatic recognition of basketball goals.A basketball goal is randomly selected, and the proposed method is used to collect the infrared spectrum image of basketball goal and preprocess the collected image. The results are shown in Figure3.Figure 3
(a) Basketball goal image acquisition. (b) Basketball goal image after preprocessing.
(a)(b)According to Figure3, the proposed method can effectively collect the basketball goal image; after the completion of denoising, enhancement, and other preprocessing, it can obtain a clearer basketball goal image, which can provide strong data support for the later basketball goal recognition.Taking the manual counting result as the actual basketball goal value, the basketball goal value identified by the proposed method is compared with the actual basketball goal value. The comparison results are shown in Table3.Table 3
Comparison results.
Actual basketball goals scored (A)The number of basketball goals identified (A)Error rate (%)880.0012120.0014140.0017170.0020200.0022220.0025244.0026253.8528273.5730293.33According to Table3, the difference between the automatic recognition results of basketball goals by using the proposed method and the actual basketball goals is small, and the maximum error rate is 4%, which shows that the proposed method can effectively identify the number of basketball goals and has high accuracy.In order to evaluate the recognition performance of the proposed method intuitively, the method in reference [7] (moving target detection and recognition method based on image sequence) and the method in reference [8] (image adaptive target recognition method based on deep feature learning) are selected as comparison methods. The basketball goal recognition results of the three methods are shown in Table 4.Table 4
Basketball goal recognition results of three methods.
The number of basketball goals (A)Recognition methodsIdentification results (A)Accuracy (%)Mean identification time (s)IdentifiedNot identified6Method of this paper60100.05.03Methods in reference [7]60100.010.98Methods in reference [8]60100.010.2511Method of this paper110100.05.17Methods in reference [7]110100.013.05Methods in reference [8]10187.112.9319Method of this paper190100.06.09Methods in reference [7]18194.715.98Methods in reference [8]17289.516.1325Method of this paper24196.06.53Methods in reference [7]23292.017.18Methods in reference [8]22388.017.8130Method of this paper29196.06.97Methods in reference [7]26486.719.26Methods in reference [8]24680.020.31According to Table4, with the increasing number of basketball goals, the recognition accuracy of the three methods has declined, and the decline of this method is significantly lower than that of the other two methods. The average accuracy of the proposed method is 98.4%, and the other two methods are 94.7% and 88.9%, respectively. The average recognition time of the proposed method is significantly lower than that of the other two methods. The reason is that this method obtains the optimal result with the least steps through the adaptive convolution neural network with good generalization ability, which saves the time of automatic recognition of basketball goals. It is proved that the recognition accuracy of the proposed method is significantly higher than that of the comparison method, and the automatic recognition time of basketball goals is the shortest.The accuracy of the three methods for basketball goal recognition is compared when adding different additive noise. The results are shown in Table5.Table 5
Basketball goal recognition results under different noises.
Noise (dB)Recognition methodThe number of basketball goals recognized (A)Accuracy (%)10Method of this paper30100.00Methods in reference [7]2893.33Methods in reference [8]2790.0020Method of this paper30100.00Methods in reference [7]2893.33Methods in reference [8]2790.0030Method of this paper30100.00Methods in reference [7]2790.00Methods in reference [8]2686.6640Method of this paper2996.66Methods in reference [7]2686.66Methods in reference [8]2480.0050Method of this paper2686.67Methods in reference [7]2273.33Methods in reference [8]1963.33According to Table5, with the increasing of additive noise, the basketball goal recognition rate of the three methods has decreased. The method in this paper is least affected by noise because wavelet transform and analysis are used to denoise and enhance the collected basketball goal image. The results show that under the influence of different noises, the proposed method is least affected by noise and has the highest accuracy of basketball goal recognition.In order to accurately measure the recognition accuracy of different methods for basketball goals, the average accuracy of each method is used to measure the basketball goal recognition results. The results are shown in Table6.Table 6
Comparison results of average precision.
The number of basketball goals (A)Mean accuracy (%)Method of this paperMethods in reference [7]Methods in reference [8]399.5396.6395.01699.4696.2295.09999.0896.6595.141299.1196.6495.021599.2396.7195.451899.0296.8295.842199.1195.9195.932499.2495.8695.982799.3895.8595.263099.2795.6794.34It can be seen from Table6 that the average accuracy of the proposed method is significantly higher than that of the methods in reference [7] and reference [8]. The average accuracy of the proposed method to identify basketball goals has been kept above 99%, which shows that this algorithm has high performance of basketball goal recognition, and the basketball goal recognition effect is good.The basketball goal recognition errors of the three methods under different color backgrounds are tested, and the test results are shown in Table7.Table 7
Recognition error under different color backgrounds.
Basketball goal background environment colorColorMethod of this paperMethods in reference [7]Methods in reference [8]PureWhite000Yellow00.110.10Blue00.120.10Red00.130.12Green00.160.15ComplexYellow, Blue0.080.240.30Yellow, Red0.100.500.55White, Yellow, Green0.220.930.90White, Yellow, Green, Red0.251.551.45White, Yellow, Blue, Red, Green0.322.652.60Analysis of Table7 shows that when the background color is monochrome, the proposed method can accurately identify all basketball goals, and the recognition rate is 100%. Under the complex background color, the highest basketball goal recognition error of the proposed method is 0.32, which is far lower than the methods in reference [7] and reference [8]. It shows that this method has high recognition rate of basketball goal in different background environments, and the recognition error is low.The results are be shown in Figure4. As shown in Figure 4, the values of the yellow bar represent the results of our proposal, and the others are denoted in notations in Figure 4. And on all datasets, we can see that our proposal is better than others except D4 (which represents the goal is 25), which all methods obtain the same results. It indicates that our proposal can perform well than the other three methods.Figure 4
Comparison of the results of all methods with that of basketball goal predictions.


## 4. Conclusions

This paper studies the basketball goal recognition method based on image processing and improved algorithm and realizes the basketball goal recognition based on convolution neural network, but there are still some problems to be solved. Although our method has achieved good prediction accuracy at present compared with other popular methods, it is still unable to achieve considerable accuracy in the face of complex basketball games environment, and the training time of the model is long, which cannot meet the purpose of real-time predict. In the future, we will further optimize the model to improve the training speed while ensuring the accuracy. Furthermore, the research work will continue from the following aspects:(1)
The performance of basketball goal recognition should be improved based on image processing and improved algorithm, the design of deep convolution network with higher information recognition accuracy under limited sample data should be optimized, and the real-time performance of the algorithm should be improved.(2)
Considering the application of many technologies in the real-time collection task of basketball goals, we can continuously perceive the movement state of basketball.



---

*Source: 9996736-2021-06-07.xml*