# Tennis Video Target Tracking Based on Mobile Network Communication and Machine Learning Algorithm

Authors: Fanming Meng
Journal: International Transactions on Electrical Energy Systems
 (2022)

---

## Abstract

The application of information technology has realized the transformation of people’s production and lifestyle and also promoted the development of the sports industry. At present, the application of informatization in the field of sports is becoming more and more powerful. By using the advanced methods and technologies of its information display, this paper aimed to realize the research on the tracking of tennis sports video objects in the mobile network environment. It is helpful to analyze and solve the objectivity problems such as most of the loopholes in the single research and traditional methods of tennis video target tracking research. By drawing on the principles and rules of machine learning algorithms, the tennis video target tracking research is carried out, and the informatization and dataization of tennis are realized. In the experiment of the target tracking algorithm, 12 tracking videos have higher tracking accuracy than other parameters. The overall tracking accuracy of the video sequence under grayscale feature was 0.694; the tracking accuracy was the highest, but the tracking speed was the lowest compared to other parameters in the experiment. Therefore, it is very important to study the target tracking of tennis sports in the tennis field.


---

## Body

## 1. Introduction

With the continuous improvement of the level of competitive ball events such as tennis, table tennis, and football, in extreme cases, it is difficult for referees on the court to make reliable and accurate judgments on ball out-of-bounds and violations only by relying on naked eyes. In addition, the tennis target tracking system can improve the viewing of tennis events and accurately reproduce wonderful moments. The athlete’s batting trajectory obtained by the target tracking system can quantify the ability of the athlete, find out the insufficiency of one’s and the enemy’s long-distance mobilization ability, conduct targeted scientific training, and make strategies for winning the enemy. The tactical strategies behind popular sports video games are also derived from the analysis of the results of human competitions and then the application of the rules to the game to improve the player’s sense of realism and immersion. Therefore, the research on tennis target tracking system is of great application value. Starting from the tennis target tracking problem, this paper studied related problems including tennis contour detection, field coordinate calculation, tennis target tracking recognition, and drop point prediction. The ultimate goal is to develop a tennis target tracking system with high reliability, high precision, and low cost and make some contributions to the development of tennis through the research and realization of the tennis target tracking system.The tennis target tracking system can overcome the defect that humans rely on the naked eye to observe, assist referees to make more accurate judgments, effectively curb the referees’ motivation to operate in the dark, and ensure the fairness of sports competition. However, the complexity of the current tennis target tracking system and the high deployment cost limit its application. The main research contents of tennis target tracking system include accurate extraction of tennis target tracking, high-precision calibration of vision system, high-precision calculation of tennis court coordinates, track point recognition for tennis target tracking, trajectory reconstruction, and accurate prediction of tennis landing point coordinates. This paper mainly introduced the tennis target tracking system, including tennis contour extraction, coordinate transformation, tennis target tracking recognition, and drop prediction algorithms, and the realization of modules such as 3D reconstruction of tennis target tracking. First, the implementation frame diagram of the tennis target tracking system was introduced. Then, the functional testing experiments of software modules such as 3D reconstruction were introduced. Finally, the feasibility of the tennis target tracking system designed in this paper was verified by a complete tennis target tracking experiment.Tracking research on targets in tennis videos is currently the focus of research in the field of sports, and many scholars have studied it at the moment; for example, Kovalchik and Ingram developed a Monte Carlo game simulation method for analyzing target tracking in tennis sports given a specific game format [1]. The purpose of Cant was to examine the ability of the physical model to estimate the trajectory rotation measured by a multi-camera ball tracking system. The ball rotation rate and rotation axis estimated from a theoretical ball trajectory model were evaluated for high-speed vision (ground truth) accuracy [2]. Baughman et al. proposed a customized machine learning algorithm that provides a learning mechanism to enable the ASR system to adapt to the sports field [3]. Ghosh et al. aimed to predict the outcome of tennis singles matches using the UCI database of eight Grand Slam tennis tournaments and used various metrics such as root mean square error, accuracy, false positive rate, true positive rate, and kappa to evaluate the classification performance [4]. Gong and Wang aimed to create tagging and indexing of video content to facilitate user acquisition through computer processing, analysis, and understanding of video content. Video tennis classification has high research and application value [5]. However, due to site constraints or equipment reasons, the above research is only in the theoretical stage, and it cannot be put into practice without sufficient experimental data and sophisticated equipment.It is a very novel method to use machine learning algorithm to study tennis video target tracking; for example, Xiao proposed that the information sources of competitors in tennis are usually different technical and tactical play styles, which can be seen from the video of tennis historical matches [6]. In order to explore the development of college tennis in the era of big data, Hu used methods such as literature research, logical analysis, interviews, and literature surveys to study tennis videos [7]. Wang and Zhang proposed related research on the real-time evaluation algorithm of human motion of tennis training robot, in order to make up for the deficiency in this field [8]. Jie and Fei proposed a method combining tennis player detection with an improved Gaussian mixture model to remove the shadows of players in tennis matches [9]. Grundstein and Cooper proposed to use mobile communication network and robotics technology to help the identification and tracking of tennis video, and their accuracy can help tennis develop better [10]. However, the current research on target tracking technology in the context of mobile communication networks still does not get rid of the definition and thinking based on traditional tennis and also lacks in-depth analysis and discussion of the functionality of machine learning algorithms. This also hinders the highly integrated and advantageous play of mobile network communication technology and tennis. Further research is still needed to tap the potential of mobile network communication technology, improve the research environment, and improve the research level.The innovations of this paper are as follows: (1) The paper implemented a tennis contour extraction algorithm based on three-frame difference, local dynamic threshold, and improved YOLOv4 model. The tennis outline was roughly extracted in real time by the three-frame difference method. When the initial trajectory of the tennis was detected, the tennis outline was re-detected by the local dynamic threshold to obtain a more accurate tennis outline. For the tennis contour with a large area, the improved YOLOv4 model was used to judge the contour area to reduce the probability of incorrectly identifying the tennis ball. (2) The paper realized a calculation method of the transformation matrix of the site coordinate system based on the characteristic points of the calibration plate, fixed points of the site, and RANSAC. This method improved the problem that the measurement error of the distant tennis coordinates is large due to the fact that the calibration point of the calibration plate is not on the plane of the tennis court when the calibration plate is used alone to calibrate the court. At the same time, the calibration plate area detection algorithm was optimized, and the reliability of the traditional calibration plate area detection algorithm was improved. (3) The paper established a dynamic model of tennis and implemented a tennis target tracking and identification algorithm. The algorithm evaluates the trajectory candidate points through four aspects: the movement direction, movement speed, movement acceleration, and coplanarity of the tennis target tracking point. And the idea of full connection was used for trajectory matching and reconstruction, which realized accurate and real-time recognition of tennis target tracking. At the same time, a tennis landing prediction algorithm based on UKF was implemented, which could accurately predict the landing coordinates that meet the accuracy requirements.


## 2. Tennis Video Target Tracking System

### 2.1. Tennis Target Tracking System

The core technology of the tennis target tracking measurement system consists of five parts: accurate detection and identification of tennis contours in the game court space, high-precision calibration of monocular and binocular cameras, high-precision solution of court coordinate transformation matrix, identification of tennis target tracking points, and trajectory reconstruction [11], as shown in Figure 1.Figure 1
Structure of tennis video target tracking system.As can be seen from Figure1, the deployment cost of the tennis target tracking measurement system is a problem. The high cost of tennis target tracking system is an important reason that limits its popular application. In order to achieve higher accuracy, the system is generally equipped with multiple high frame rate and high resolution cameras, resulting in a significant increase in cost. In order to reduce the number of cameras and reduce the cost of the measurement system, this paper intended to use binocular cameras to measure the high-precision coordinates of tennis balls, as shown in Table 1.Table 1
Tennis target tracking measurement system deployment costs.
System nameSystem configurationCostSony Super Camera System10 high-speed cameras$70,000Israel PlaySight Hawk-Eye System6 cameras$12,000China Network Track Measurement SystemUnknown$220,000Beijing Rui Gai Technology Eagle Eye System8 cameras, 2 video cameras260,000 RMBIt can be seen from Table1 that there is a trade-off between the reliability, accuracy, and real-time indicators of the measurement system. The reliability and accuracy indicators of the key technology of the tennis measurement system contradict its real-time indicators. The higher the reliability and accuracy, the higher the algorithm complexity, and thus the lower the real-time performance. Therefore, it is necessary to make trade-offs in algorithm design [12, 13].


### 2.2. Image Preprocessing

When the camera captures an image, the interference is caused by reasons such as the noise in the process of CMOS/CCD sensor converting optical signal to digital signal, errors generated by the system during the acquisition of images, smear from objects moving at high speed, camera pose shakes, and flickering lights in the environment, which may cause the output of the vision system to shake when the algorithm is the same [14]. Therefore, it is necessary to reduce the influence of noise errors, eliminate invalid error information, and restore useful image details in the image. Noise refers to information whose distribution and size are unpredictable. The effect of various types of filters on tennis images is shown in Figure 2.Figure 2
The effect of various types of filters on images containing tennis balls (a) Original image, (b) Bilateral filter, (c) Mean filter, (d) Median filter, (e) Gaussian filter.
(a)(b)(c)(d)(e)It can be seen from Figure2 that the binarized image will make the processing algorithm simpler in the solution of specific vision problems. The setting method of the fixed threshold is relatively simple, the amount of calculation is small, and the effect is good in an environment with stable light, but it needs to be manually set and debugged repeatedly to obtain the optimal effect. The adaptive threshold does not need to be manually set repeatedly, but it is computationally intensive and unstable. The open algorithm can be used to remove small blocks of noise, and the closed algorithm can be used to fill small black blocks in the binary image.


### 2.3. Background Difference Method

In the tennis target tracking measurement system, the position of the camera and the attitude of the observed scene are fixed, and the background difference method is a commonly used and reliable tennis contour extraction algorithm. The process is shown in Figure3. The application of the background difference method requires the design of two basic links: foreground detection rules, and background modeling and rule update according to the lighting conditions of the site [15].Figure 3
The frame difference method detecting the outline of the tennis ball.As can be seen from Figure3, when the foreground detection algorithm is initialized, it needs to collect an image that does not contain tennis balls from the left and right cameras and an image of the athlete as the background image and then obtain the difference map by taking the difference of each frame image. The contour area of the tennis ball is obtained by processing the difference map according to formula.(1)fx,y=1,It−It−1≥T,0,It−It−1T.Among them,T is the threshold, and f (x, y) indicates that the pixel at (x, y) is a point on the surface of the sports tennis ball. For scenes where the brightness of ambient light changes frequently (such as flashing lights, lights of passing cars at night, and lights of spectators’ mobile phones) and the ambient background includes spectators and other nonstationary objects, the tennis contour detection result based on background difference will contain many wrong tennis contours, which will have a great impact on the accuracy of the subsequent tennis target tracking point recognition algorithm.


### 2.4. Optical Flow Method

The basic idea of optical flow is to represent the instantaneous movement changes of the pixels of moving objects in a grayscale image. The principle of tennis contour extraction based on optical flow is to design a feature descriptor corresponding to the tennis contour according to the characteristics of the tennis contour when processing the images collected by the binocular camera to detect all points that satisfy the tennis contour characteristics. A motion speed and direction are assigned to all feature points individually, so as to obtain the corresponding motion vector field. The process of tracking the outline of a tennis ball through the optical flow method is divided into two parts.The designed image contour feature descriptor sub-parameters were tested repeatedly through experiments, so that the feature point description can meet the requirements of reliable detection of the initial feature points contained in the edge of the tennis ball.To track the selected feature points, since there are many points in the court environment that satisfy the descriptors of the tennis contour feature points, the feature points need to be tracked by the optical flow method; the process subdivision link of the tennis contour extraction and the detection effect are shown in Figure4.Figure 4
Optical flow method for tennis video object detection.It can be seen from Figure4 that it is determined whether the point is stationary in the next frame. If the feature point is in a stationary state, this means that the feature point is a point on the surface of a stationary object, not a moving tennis outline point, which needs to be deleted. Only the moving feature points located on the tennis contour are tracked, and finally the region classification is performed according to the moving feature points to obtain all the suspected tennis contour regions [16].As a result, the picture sequence that is thought through is input, and the background difference method and the optical flow method are used to extract the outline of the high-speed tennis ball. The accuracy and time consumption of the two are shown in Table2.Table 2
Tennis contour detection results of each algorithm.
MethodBackground differenceOptical flow methodX coordinate7565Y coordinate5438Error (pixel)12.16334.114Time (ms)520It can be seen from Table2 that the feature points generated by the optical flow method are all Harris corner points in the entire image due to initialization. With the operation of the system, the results of the optical flow algorithm are gradually stable, and the processing time of a single frame of the optical flow algorithm is very long. In the tennis target tracking measurement system, the real-time requirement of the tennis contour is very high, so the optical flow method is not suitable for the detection of the high-speed tennis contour. The advantage of the background difference method is that the speed is fast. Since the target tracking used to calculate the tennis coordinates includes two tennis balls, the calculated tennis center coordinates are the center of the tennis target tracking in the two frames before and after, and the deviation from the current frame tennis target tracking center coordinates is not large. Therefore, when building a tennis target tracking system, it is most appropriate to use the background difference method for process construction.


### 2.5. Kernel Correlation Filtering Algorithm Based on Machine Learning

In order to solve the problem of sparse samples, KCF uses a cyclic matrix to increase the number of samples. According to the characteristic that the circulant matrix can be diagonalized in the Fourier domain, the convolution calculation in the time domain is converted into the multiplication of the corresponding elements in the frequency domain to improve the running speed of the algorithm. In order to deeply understand the KCF algorithm, the expression of KCF in a one-dimensional vector is derived here:(2)Xn=Pn,n=0,1,2...n−1,(3)P=0001100001000010.In (2) and (3), P is the cyclic shift factor, which is the training sample after the square matrix Xn is shifted n times, and Xn=x0,x1,x2,…,xn, where x is a one-dimensional column vector, which represents the sample, and n is the number of cyclic shifts.The training samples (x, y) are obtained after cyclic shift, assuming the linear model is as follows:(4)fz=wTz.It can be infinitely close to the model law of the current sample. The corresponding constraint equation is obtained by using the least squares method to solve the parameterw:(5)minw∑ifxi−yi2+λw2.Therefore, it can be expressed as follows:(6)minw∑ifxi−yi2+λw2=minwXw−y2+λw2.Let the derivative of (4) be 0, and the obtained expression can obtain the solution of the equation. The solution process is as follows:(7)2XTXw−y+2wλ=0,(8)w=XTX+λI−1XTy.Since the subsequent Fourier transform needs to be calculated and operated in the complex frequency domain, it is converted into the complex domain form:(9)w=XHX+λI−1XHy.In (9), H is the conjugate transpose of the matrix [17]. According to (9), the motion model parameters of the moving target can be obtained. However, according to the results given, it is necessary to invert the matrix. Although it can be solved quickly for simple data processing, it will lead to complicated operations and long processing time for complex matrices. Since the circulant matrix has the following characteristics, the parameter solving process can be simplified.(10)Xw=y⇒x⊗w⇒FxFw=Fy,(11)X=Fdiagx⌢FH.In (11), diag() is the diagonal matrix of x, which can be obtained by substituting it into (10):(12)w=x⊗yx⊗x+λ.The definition in (12) is the element-to-element multiplication. The above is the model processing method for training models in linear space. Extending the application of circulant matrix to solving models in nonlinear space requires a nonlinear mapping function ϕxi to convert nonlinear space into linear space. According to (10), w is related to the sample, so it is assumed that w has the following relationship with the sample:(13)w=∑aiϕxi.In (11), the solution of w from the original equation is transformed into the solution of a, and data processing is performed in the dual space of w; then, a can be deduced:(14)a=φXφXT+λI−1y,(15)a=K+λI−1y.In (15), K=φXφXT is the kernel matrix of the kernel space, which represents the correlation between samples. In order to simplify the solution process of a, K must be a cyclic matrix, so the Gaussian kernel function is selected here. Therefore, a can be simplified as follows:(16)a=FiagKxx+λ−1FHy,(17)a⌢=y⌢K⌢+xxλ.K is a cyclic matrix, and the correlation between sample pairs is calculated in the Gaussian kernel function, which is expressed as follows:(18)Kxx=φTxφx′.Among them, the Gaussian kernel function is defined as follows:(19)Kxx′=exp−1σx2+x′2−2F−1.In (19), F−1 is the inverse Fourier transform, Kxx′ is the discrete Fourier transform, x is the complex conjugate,σ is the width of the Gaussian kernel, and x′ is the feature dimension extracted from the tracking target. The above is the parameter expression process of the model, and then the specific expression of the model is obtained.From the above algorithm description, it can be seen that the kernel correlation filtering algorithm plays an important role in the study of tennis video target extraction, and its derived correlation with the Gaussian kernel function can make the image clearer in the target tracking video.


### 2.6. Moving Target Extraction Based on Correlation Filtering Algorithm

After the background image is acquired, subtract the current frame from its corresponding background frame. In addition to the experiment of moving target extraction based on background difference subtraction in slow motion, the corresponding experiments are also carried out in the competition and close-up shots [18, 19]. Figure 5 shows a frame of the original image selected in each shot, corresponding to the background image generated by GMM.Figure 5
Effect of moving target extraction based on background difference method (a) Original image, (b) Background image, (c) Foreground image.
(a)(b)(c)As can be seen from Figure5, the automatic generation and update of background images using GMM are not ideal. The extracted foreground target is basically similar to the original image content, including both real moving targets (such as players and balls) and background parts (such as audience and field lines), but the scoreboard is fixed due to its position and size, which can be basically removed in the foreground target. The reason for the analysis is mainly because the camera is not fixed relative to the monitoring scene during the shooting of the game, and the background is a moving background rather than a dynamic background; that is, in addition to the target movement in the two frames before and after, there is also relative movement in the background. Therefore, the parts that should belong to the background, such as the audience and the field line, are also mistaken for the moving target. In addition, a tennis video consists of multiple shots, the shots are frequently changed, and the number of frames in a single shot or in a scene is not enough, so the update cycle of the background is not long enough, and the situation shown in Figure 5(a) often occurs (the game scene appears before the close-up shot is updated), which is also one of the reasons for the inaccuracy of the background image [20].


## 2.1. Tennis Target Tracking System

The core technology of the tennis target tracking measurement system consists of five parts: accurate detection and identification of tennis contours in the game court space, high-precision calibration of monocular and binocular cameras, high-precision solution of court coordinate transformation matrix, identification of tennis target tracking points, and trajectory reconstruction [11], as shown in Figure 1.Figure 1
Structure of tennis video target tracking system.As can be seen from Figure1, the deployment cost of the tennis target tracking measurement system is a problem. The high cost of tennis target tracking system is an important reason that limits its popular application. In order to achieve higher accuracy, the system is generally equipped with multiple high frame rate and high resolution cameras, resulting in a significant increase in cost. In order to reduce the number of cameras and reduce the cost of the measurement system, this paper intended to use binocular cameras to measure the high-precision coordinates of tennis balls, as shown in Table 1.Table 1
Tennis target tracking measurement system deployment costs.
System nameSystem configurationCostSony Super Camera System10 high-speed cameras$70,000Israel PlaySight Hawk-Eye System6 cameras$12,000China Network Track Measurement SystemUnknown$220,000Beijing Rui Gai Technology Eagle Eye System8 cameras, 2 video cameras260,000 RMBIt can be seen from Table1 that there is a trade-off between the reliability, accuracy, and real-time indicators of the measurement system. The reliability and accuracy indicators of the key technology of the tennis measurement system contradict its real-time indicators. The higher the reliability and accuracy, the higher the algorithm complexity, and thus the lower the real-time performance. Therefore, it is necessary to make trade-offs in algorithm design [12, 13].


## 2.2. Image Preprocessing

When the camera captures an image, the interference is caused by reasons such as the noise in the process of CMOS/CCD sensor converting optical signal to digital signal, errors generated by the system during the acquisition of images, smear from objects moving at high speed, camera pose shakes, and flickering lights in the environment, which may cause the output of the vision system to shake when the algorithm is the same [14]. Therefore, it is necessary to reduce the influence of noise errors, eliminate invalid error information, and restore useful image details in the image. Noise refers to information whose distribution and size are unpredictable. The effect of various types of filters on tennis images is shown in Figure 2.Figure 2
The effect of various types of filters on images containing tennis balls (a) Original image, (b) Bilateral filter, (c) Mean filter, (d) Median filter, (e) Gaussian filter.
(a)(b)(c)(d)(e)It can be seen from Figure2 that the binarized image will make the processing algorithm simpler in the solution of specific vision problems. The setting method of the fixed threshold is relatively simple, the amount of calculation is small, and the effect is good in an environment with stable light, but it needs to be manually set and debugged repeatedly to obtain the optimal effect. The adaptive threshold does not need to be manually set repeatedly, but it is computationally intensive and unstable. The open algorithm can be used to remove small blocks of noise, and the closed algorithm can be used to fill small black blocks in the binary image.


## 2.3. Background Difference Method

In the tennis target tracking measurement system, the position of the camera and the attitude of the observed scene are fixed, and the background difference method is a commonly used and reliable tennis contour extraction algorithm. The process is shown in Figure3. The application of the background difference method requires the design of two basic links: foreground detection rules, and background modeling and rule update according to the lighting conditions of the site [15].Figure 3
The frame difference method detecting the outline of the tennis ball.As can be seen from Figure3, when the foreground detection algorithm is initialized, it needs to collect an image that does not contain tennis balls from the left and right cameras and an image of the athlete as the background image and then obtain the difference map by taking the difference of each frame image. The contour area of the tennis ball is obtained by processing the difference map according to formula.(1)fx,y=1,It−It−1≥T,0,It−It−1T.Among them,T is the threshold, and f (x, y) indicates that the pixel at (x, y) is a point on the surface of the sports tennis ball. For scenes where the brightness of ambient light changes frequently (such as flashing lights, lights of passing cars at night, and lights of spectators’ mobile phones) and the ambient background includes spectators and other nonstationary objects, the tennis contour detection result based on background difference will contain many wrong tennis contours, which will have a great impact on the accuracy of the subsequent tennis target tracking point recognition algorithm.


## 2.4. Optical Flow Method

The basic idea of optical flow is to represent the instantaneous movement changes of the pixels of moving objects in a grayscale image. The principle of tennis contour extraction based on optical flow is to design a feature descriptor corresponding to the tennis contour according to the characteristics of the tennis contour when processing the images collected by the binocular camera to detect all points that satisfy the tennis contour characteristics. A motion speed and direction are assigned to all feature points individually, so as to obtain the corresponding motion vector field. The process of tracking the outline of a tennis ball through the optical flow method is divided into two parts.The designed image contour feature descriptor sub-parameters were tested repeatedly through experiments, so that the feature point description can meet the requirements of reliable detection of the initial feature points contained in the edge of the tennis ball.To track the selected feature points, since there are many points in the court environment that satisfy the descriptors of the tennis contour feature points, the feature points need to be tracked by the optical flow method; the process subdivision link of the tennis contour extraction and the detection effect are shown in Figure4.Figure 4
Optical flow method for tennis video object detection.It can be seen from Figure4 that it is determined whether the point is stationary in the next frame. If the feature point is in a stationary state, this means that the feature point is a point on the surface of a stationary object, not a moving tennis outline point, which needs to be deleted. Only the moving feature points located on the tennis contour are tracked, and finally the region classification is performed according to the moving feature points to obtain all the suspected tennis contour regions [16].As a result, the picture sequence that is thought through is input, and the background difference method and the optical flow method are used to extract the outline of the high-speed tennis ball. The accuracy and time consumption of the two are shown in Table2.Table 2
Tennis contour detection results of each algorithm.
MethodBackground differenceOptical flow methodX coordinate7565Y coordinate5438Error (pixel)12.16334.114Time (ms)520It can be seen from Table2 that the feature points generated by the optical flow method are all Harris corner points in the entire image due to initialization. With the operation of the system, the results of the optical flow algorithm are gradually stable, and the processing time of a single frame of the optical flow algorithm is very long. In the tennis target tracking measurement system, the real-time requirement of the tennis contour is very high, so the optical flow method is not suitable for the detection of the high-speed tennis contour. The advantage of the background difference method is that the speed is fast. Since the target tracking used to calculate the tennis coordinates includes two tennis balls, the calculated tennis center coordinates are the center of the tennis target tracking in the two frames before and after, and the deviation from the current frame tennis target tracking center coordinates is not large. Therefore, when building a tennis target tracking system, it is most appropriate to use the background difference method for process construction.


## 2.5. Kernel Correlation Filtering Algorithm Based on Machine Learning

In order to solve the problem of sparse samples, KCF uses a cyclic matrix to increase the number of samples. According to the characteristic that the circulant matrix can be diagonalized in the Fourier domain, the convolution calculation in the time domain is converted into the multiplication of the corresponding elements in the frequency domain to improve the running speed of the algorithm. In order to deeply understand the KCF algorithm, the expression of KCF in a one-dimensional vector is derived here:(2)Xn=Pn,n=0,1,2...n−1,(3)P=0001100001000010.In (2) and (3), P is the cyclic shift factor, which is the training sample after the square matrix Xn is shifted n times, and Xn=x0,x1,x2,…,xn, where x is a one-dimensional column vector, which represents the sample, and n is the number of cyclic shifts.The training samples (x, y) are obtained after cyclic shift, assuming the linear model is as follows:(4)fz=wTz.It can be infinitely close to the model law of the current sample. The corresponding constraint equation is obtained by using the least squares method to solve the parameterw:(5)minw∑ifxi−yi2+λw2.Therefore, it can be expressed as follows:(6)minw∑ifxi−yi2+λw2=minwXw−y2+λw2.Let the derivative of (4) be 0, and the obtained expression can obtain the solution of the equation. The solution process is as follows:(7)2XTXw−y+2wλ=0,(8)w=XTX+λI−1XTy.Since the subsequent Fourier transform needs to be calculated and operated in the complex frequency domain, it is converted into the complex domain form:(9)w=XHX+λI−1XHy.In (9), H is the conjugate transpose of the matrix [17]. According to (9), the motion model parameters of the moving target can be obtained. However, according to the results given, it is necessary to invert the matrix. Although it can be solved quickly for simple data processing, it will lead to complicated operations and long processing time for complex matrices. Since the circulant matrix has the following characteristics, the parameter solving process can be simplified.(10)Xw=y⇒x⊗w⇒FxFw=Fy,(11)X=Fdiagx⌢FH.In (11), diag() is the diagonal matrix of x, which can be obtained by substituting it into (10):(12)w=x⊗yx⊗x+λ.The definition in (12) is the element-to-element multiplication. The above is the model processing method for training models in linear space. Extending the application of circulant matrix to solving models in nonlinear space requires a nonlinear mapping function ϕxi to convert nonlinear space into linear space. According to (10), w is related to the sample, so it is assumed that w has the following relationship with the sample:(13)w=∑aiϕxi.In (11), the solution of w from the original equation is transformed into the solution of a, and data processing is performed in the dual space of w; then, a can be deduced:(14)a=φXφXT+λI−1y,(15)a=K+λI−1y.In (15), K=φXφXT is the kernel matrix of the kernel space, which represents the correlation between samples. In order to simplify the solution process of a, K must be a cyclic matrix, so the Gaussian kernel function is selected here. Therefore, a can be simplified as follows:(16)a=FiagKxx+λ−1FHy,(17)a⌢=y⌢K⌢+xxλ.K is a cyclic matrix, and the correlation between sample pairs is calculated in the Gaussian kernel function, which is expressed as follows:(18)Kxx=φTxφx′.Among them, the Gaussian kernel function is defined as follows:(19)Kxx′=exp−1σx2+x′2−2F−1.In (19), F−1 is the inverse Fourier transform, Kxx′ is the discrete Fourier transform, x is the complex conjugate,σ is the width of the Gaussian kernel, and x′ is the feature dimension extracted from the tracking target. The above is the parameter expression process of the model, and then the specific expression of the model is obtained.From the above algorithm description, it can be seen that the kernel correlation filtering algorithm plays an important role in the study of tennis video target extraction, and its derived correlation with the Gaussian kernel function can make the image clearer in the target tracking video.


## 2.6. Moving Target Extraction Based on Correlation Filtering Algorithm

After the background image is acquired, subtract the current frame from its corresponding background frame. In addition to the experiment of moving target extraction based on background difference subtraction in slow motion, the corresponding experiments are also carried out in the competition and close-up shots [18, 19]. Figure 5 shows a frame of the original image selected in each shot, corresponding to the background image generated by GMM.Figure 5
Effect of moving target extraction based on background difference method (a) Original image, (b) Background image, (c) Foreground image.
(a)(b)(c)As can be seen from Figure5, the automatic generation and update of background images using GMM are not ideal. The extracted foreground target is basically similar to the original image content, including both real moving targets (such as players and balls) and background parts (such as audience and field lines), but the scoreboard is fixed due to its position and size, which can be basically removed in the foreground target. The reason for the analysis is mainly because the camera is not fixed relative to the monitoring scene during the shooting of the game, and the background is a moving background rather than a dynamic background; that is, in addition to the target movement in the two frames before and after, there is also relative movement in the background. Therefore, the parts that should belong to the background, such as the audience and the field line, are also mistaken for the moving target. In addition, a tennis video consists of multiple shots, the shots are frequently changed, and the number of frames in a single shot or in a scene is not enough, so the update cycle of the background is not long enough, and the situation shown in Figure 5(a) often occurs (the game scene appears before the close-up shot is updated), which is also one of the reasons for the inaccuracy of the background image [20].


## 3. Tennis Video Target Extraction Experiment

### 3.1. Target Extraction Algorithms

In this paper, the machine learning algorithm was used to extract the features of moving targets. At the same time, in order to deal with the impact of illumination changes on target tracking, this paper extracted the grayscale features of the tracking targets. Therefore, a total of 42 dimensional features were extracted in this paper, and the feature fusion algorithm can fully describe the appearance features of moving objects and the local tracking feature information of moving objects. For 26 occlusion video experiments, setting the occlusion detection parameters toλ1=0.4andλ2=0.3 can not only ensure that the method in this paper can recognize partial occlusion and update the model, but also quickly capture the moving target and retrack it in the case of full occlusion [21]. Figure 6 is the data record of the parameters adjusted in this experiment. Since the parameter adjustment requires multiple experiments with different parameter values in the same sequence, the amount of data is large, so only part of the data is listed here.Figure 6
Statistical plot of data logging for experimental tuning parameters.It can be seen from Figure6 that when the algorithm is λ1=0.4andλ2=0.3, 12 tracking videos have higher tracking accuracy than other parameters. The overall tracking accuracy of the video sequence under grayscale feature parameter is 0.694, and the tracking accuracy is the highest. But the tracking speed is the lowest compared to other parameters. According to the analysis of the occlusion detection algorithm, it can be seen that the occlusion detection mechanism determines that there is currently occlusion, and at this time, the machine learning algorithm is executed, and the candidate frame is not generated. However, if the current index parameters are selected too much, the response value of the target will be mistakenly detected as occlusion, resulting in inaccurate tracking [22]. Therefore, the parameters selected in this paper were λ1=0.4andλ2=0.3. In order to make the results of this paper more convincing, the tracking accuracy was analyzed by using the position error, and the data with excellent performance was counted. Here, some of the video results are listed in Figure 7(a), and the tracking speed data is listed in Figure 7(b). It can be seen from Figure 7 that the KCF, fDSST, OCTKCF, and KCFDPT algorithms perform better, so occlusion detection algorithm and machine learning algorithm were compared with these four algorithms.Figure 7
Comparison of the accuracy (a) and tracking speed (b) of 5 target tracking algorithms.
(a)(b)In Figure7(a), according to the number of sequence videos with good performance and the average accuracy of the corresponding algorithms in evaluating the performance of the algorithm, the performance was ranked as OCTKCF, EBKCF, KCFDPT, KCF, and fDSST, among which KCF and fDSST had comparable performance. According to the average speed data in Figure 7(b), the tracking speed was KCF, OCTKCF, EBKCF, KCFDPT, and fDSST algorithms in sequence. In the overall detection speed of moving objects, the average speed of EBKCF was 19.09FPS, which can basically achieve real-time performance. Among the compared algorithms, only the KCF algorithm can track moving targets at high speed, and the other algorithms are low in speed [23].


### 3.2. Experimental Results of Target Extraction

Two commonly used criteria for evaluating the detection effect of an algorithm are recall rate and accuracy rate. This paper also used the accuracy of boundary positioning to further measure the accuracy of the algorithm [24]. Among them, the number of missing frames refers to the frames that contain the identifier but are not detected. In this paper, 5 tennis match videos with logo conversion were selected as experimental materials, and the slow-motion detection algorithm in the machine learning algorithm and the algorithm proposed in this paper were used for experiments. The results are shown in Figure 8 and Table 3.Figure 8
Experimental results of target tracking detection algorithm.Table 3
Statistical table of experimental accuracy of target tracking detection algorithm.
Video serial numberRecall (%)Precision (%)Boundary location accuracy (%)192.096.73.0295.495.43.1386.092.74.4492.392.35.9591.891.94.6It can be seen from Figure8 and Table 3 that the slow-motion detection algorithm proposed in this paper can better detect the slow motion in the video and achieve high boundary positioning and accuracy. The main reason that affects recall is the error of shot segmentation, which results in the failure to detect individual landmark shots when detecting candidate landmark shots. This algorithm has higher boundary localization accuracy and accuracy compared with existing slow-motion detection algorithms. This is because the color autocorrelation function is used in the matching process to guarantee the spatial correlation of the images. Finally, based on the initial position of the slow-motion boundary, identifying the exact boundary position by changing the brightness during the conversion process can greatly improve the accuracy of the boundary position.


### 3.3. Target Extraction Results

After three-dimensional reconstruction is performed based on the three-frame difference results, if it is determined that the tennis ball is in the descending stage, local dynamic threshold processing is enabled. Through the maximum inscribed circle method, the outline of the tennis ball extracted from the three-frame difference is processed to obtain the center coordinates of the tennis ball. It can be found that it is greatly affected by the contour noise. Due to the deformation of some arcs in the tennis outline, the calculated center coordinates are offset. Therefore, it is necessary to improve the algorithm for calculating the center coordinates of the outline. The first-order gray moment method is used to obtain the center coordinates of the above outline. The effect is shown in Figure9.Figure 9
The result of calculating the contour center by the first-order gray moment method.It can be seen from Figure9 that the method discretizes the target tracking multiple times and performs random sampling. The center coordinates reconstructed from these sampling points are compared with the real center coordinates. The original center coordinate errors (2.156, −1.606) and the x and y coordinate errors are reduced. It can be seen that the algorithm can effectively improve the accuracy of tennis target tracking and extraction. It can also be seen that the first-order gray moment method can suppress the error in tennis target tracking and extraction and obtain more accurate center coordinates compared with the direct method and the maximum inscribed circle method. By comparing the results of the three solving methods for the center coordinates of the tennis profile, as shown in Table 4, the real coordinates were (360, 360). It can be seen that the first-order gray moment method has the best effect and can effectively suppress the influence of contour edge noise on the calculation of center coordinates compared with the direct method and the maximum inscribed circle method.Table 4
Results of the calculation method for the center coordinates of each tennis profile.
Methodx coordinatey coordinateErrorDirect method/geometric center35636923.85Maximum inscribed circle method355.8357.412.39Grayscale moment method358.92350.140.85


## 3.1. Target Extraction Algorithms

In this paper, the machine learning algorithm was used to extract the features of moving targets. At the same time, in order to deal with the impact of illumination changes on target tracking, this paper extracted the grayscale features of the tracking targets. Therefore, a total of 42 dimensional features were extracted in this paper, and the feature fusion algorithm can fully describe the appearance features of moving objects and the local tracking feature information of moving objects. For 26 occlusion video experiments, setting the occlusion detection parameters toλ1=0.4andλ2=0.3 can not only ensure that the method in this paper can recognize partial occlusion and update the model, but also quickly capture the moving target and retrack it in the case of full occlusion [21]. Figure 6 is the data record of the parameters adjusted in this experiment. Since the parameter adjustment requires multiple experiments with different parameter values in the same sequence, the amount of data is large, so only part of the data is listed here.Figure 6
Statistical plot of data logging for experimental tuning parameters.It can be seen from Figure6 that when the algorithm is λ1=0.4andλ2=0.3, 12 tracking videos have higher tracking accuracy than other parameters. The overall tracking accuracy of the video sequence under grayscale feature parameter is 0.694, and the tracking accuracy is the highest. But the tracking speed is the lowest compared to other parameters. According to the analysis of the occlusion detection algorithm, it can be seen that the occlusion detection mechanism determines that there is currently occlusion, and at this time, the machine learning algorithm is executed, and the candidate frame is not generated. However, if the current index parameters are selected too much, the response value of the target will be mistakenly detected as occlusion, resulting in inaccurate tracking [22]. Therefore, the parameters selected in this paper were λ1=0.4andλ2=0.3. In order to make the results of this paper more convincing, the tracking accuracy was analyzed by using the position error, and the data with excellent performance was counted. Here, some of the video results are listed in Figure 7(a), and the tracking speed data is listed in Figure 7(b). It can be seen from Figure 7 that the KCF, fDSST, OCTKCF, and KCFDPT algorithms perform better, so occlusion detection algorithm and machine learning algorithm were compared with these four algorithms.Figure 7
Comparison of the accuracy (a) and tracking speed (b) of 5 target tracking algorithms.
(a)(b)In Figure7(a), according to the number of sequence videos with good performance and the average accuracy of the corresponding algorithms in evaluating the performance of the algorithm, the performance was ranked as OCTKCF, EBKCF, KCFDPT, KCF, and fDSST, among which KCF and fDSST had comparable performance. According to the average speed data in Figure 7(b), the tracking speed was KCF, OCTKCF, EBKCF, KCFDPT, and fDSST algorithms in sequence. In the overall detection speed of moving objects, the average speed of EBKCF was 19.09FPS, which can basically achieve real-time performance. Among the compared algorithms, only the KCF algorithm can track moving targets at high speed, and the other algorithms are low in speed [23].


## 3.2. Experimental Results of Target Extraction

Two commonly used criteria for evaluating the detection effect of an algorithm are recall rate and accuracy rate. This paper also used the accuracy of boundary positioning to further measure the accuracy of the algorithm [24]. Among them, the number of missing frames refers to the frames that contain the identifier but are not detected. In this paper, 5 tennis match videos with logo conversion were selected as experimental materials, and the slow-motion detection algorithm in the machine learning algorithm and the algorithm proposed in this paper were used for experiments. The results are shown in Figure 8 and Table 3.Figure 8
Experimental results of target tracking detection algorithm.Table 3
Statistical table of experimental accuracy of target tracking detection algorithm.
Video serial numberRecall (%)Precision (%)Boundary location accuracy (%)192.096.73.0295.495.43.1386.092.74.4492.392.35.9591.891.94.6It can be seen from Figure8 and Table 3 that the slow-motion detection algorithm proposed in this paper can better detect the slow motion in the video and achieve high boundary positioning and accuracy. The main reason that affects recall is the error of shot segmentation, which results in the failure to detect individual landmark shots when detecting candidate landmark shots. This algorithm has higher boundary localization accuracy and accuracy compared with existing slow-motion detection algorithms. This is because the color autocorrelation function is used in the matching process to guarantee the spatial correlation of the images. Finally, based on the initial position of the slow-motion boundary, identifying the exact boundary position by changing the brightness during the conversion process can greatly improve the accuracy of the boundary position.


## 3.3. Target Extraction Results

After three-dimensional reconstruction is performed based on the three-frame difference results, if it is determined that the tennis ball is in the descending stage, local dynamic threshold processing is enabled. Through the maximum inscribed circle method, the outline of the tennis ball extracted from the three-frame difference is processed to obtain the center coordinates of the tennis ball. It can be found that it is greatly affected by the contour noise. Due to the deformation of some arcs in the tennis outline, the calculated center coordinates are offset. Therefore, it is necessary to improve the algorithm for calculating the center coordinates of the outline. The first-order gray moment method is used to obtain the center coordinates of the above outline. The effect is shown in Figure9.Figure 9
The result of calculating the contour center by the first-order gray moment method.It can be seen from Figure9 that the method discretizes the target tracking multiple times and performs random sampling. The center coordinates reconstructed from these sampling points are compared with the real center coordinates. The original center coordinate errors (2.156, −1.606) and the x and y coordinate errors are reduced. It can be seen that the algorithm can effectively improve the accuracy of tennis target tracking and extraction. It can also be seen that the first-order gray moment method can suppress the error in tennis target tracking and extraction and obtain more accurate center coordinates compared with the direct method and the maximum inscribed circle method. By comparing the results of the three solving methods for the center coordinates of the tennis profile, as shown in Table 4, the real coordinates were (360, 360). It can be seen that the first-order gray moment method has the best effect and can effectively suppress the influence of contour edge noise on the calculation of center coordinates compared with the direct method and the maximum inscribed circle method.Table 4
Results of the calculation method for the center coordinates of each tennis profile.
Methodx coordinatey coordinateErrorDirect method/geometric center35636923.85Maximum inscribed circle method355.8357.412.39Grayscale moment method358.92350.140.85


## 4. Conclusions

Tennis target tracking system plays an important role in tennis competition, but the complexity and high cost of the system limit its popularization. In this paper, a tennis target tracking system with high reliability, high precision, and relatively low cost was designed. The specific results are as follows: the extraction algorithm of tennis outline. The experimental results of the maximum inscribed circle method, the first-order gray moment, and the local dynamic threshold for the refinement of the center coordinates of the tennis contour were compared. However, the acquisition speed, resolution, and imaging quality of the camera (such as the smear problem of high-speed moving objects) will directly affect the development of the tennis trajectory measurement system. During the experiment of this system, it was found that when the speed of the tennis ball is too fast, the image of the tennis ball captured by the camera will show smear or the tennis ball distance between frames will be too large, which will lead to the reduction of the accuracy of tennis trajectory extraction. Therefore, the development of high frame rate, high resolution, and high quality cameras should be regarded as an important research direction in the field of computer vision industry.



---

*Source: 7447121-2022-09-10.xml*