# Improved Cam Shift Algorithm for Detecting Athletes' Action Targets in Sports Videos

Authors: Yi Yuan
Journal: Mathematical Problems in Engineering
 (2022)

---

## Abstract

The detection and tracking of athletes in sports videos is of great importance, as it helps to automate the analysis of sports videos, thus providing advanced tools and instruments for sports training. The Cam Shift algorithm uses the colour information of objects to achieve tracking of moving targets, so it is very important to choose a suitable colour space when obtaining the colour information of objects. To this end, this paper improves the Cam Shift algorithm by converting the visual system mechanism to perceive the colour characteristics of the image processing, i.e., converting the video image from the RGB colour space to the HSV colour space, using the H (hue) component to model the target object. Experiments show that the improved athlete detection and tracking algorithm is more robust in practical applications. The improved tracking algorithm also has better real-time performance, with a processing speed of 20 fps during the experiments.


---

## Body

## 1. Introduction

In sports videos, the movement of athletes is irregular and their posture changes in various ways as they move [1]. The colours of the athletes and the scenes may be similar, and there is often mutual occlusion between the athletes. The unique nature of sports videos poses many challenges to the process of athlete detection and tracking. Therefore, the main objective of this paper is to analyse and investigate the main current methods for motion target detection and tracking, to propose an effective algorithm for athlete detection and tracking in sports videos, and to develop a prototype system to verify the correctness of the algorithm [2].The detection and tracking of motion targets involves many disciplines such as image processing, computer vision, signal processing, and pattern recognition. In these fields, much of the processing of images and videos relies on target detection and tracking techniques, so research into motion target detection and tracking methods, and the improvement and refinement of related algorithms, is important for the advancement of these fields [3].Motion target detection and tracking can be applied to many fields such as human-computer interaction, video surveillance, video analysis, and sports training. In the field of human-computer interaction, the combination of human detection and tracking technology with other technologies such as audio can be used to develop multichannel user interfaces, which can both broaden the bandwidth of human-computer interaction and make the human-computer interaction more efficient and natural [4]. In the field of video analysis, motion target detection and tracking can be used to extract meaningful video objects from the video and analyse their behaviour, thus helping to implement content-based video retrieval techniques. Motion target detection and tracking technology can also be applied to sports video analysis; through the detection and tracking of athletes, you can obtain a variety of athletes' motion parameters and human posture parameters, which is conducive to the analysis of athletes' motion trajectory, and determine the normality of their movements, but also conducive to the application of these parameters for three-dimensional reconstruction, in a three-dimensional way to realistic simulation, design, and analysis of technical movements. It is important to improve the training effect of athletes [5].In addition, traditional machine learning algorithms are limited in their ability to process raw kinematic data, cannot effectively train on discontinuous, noisy, and high-dimensional data with missing values [6], and always require preprocessing of the raw data, including Kalman filter, fast Fourier transform (FFT), and vector coding techniques [7].It is worth noting that the balance between robustness, accuracy, and effectiveness of a competition-based computer vision motion analysis system relies on improved algorithms and hardware optimisation compared to 3D motion capture analysis in a laboratory environment [8, 9].


## 2. Related Work

In markerless action recognition based on generative algorithms, the pose shape of the human body is determined by matching the human model with information extracted from the image. For example, for a given set of model parameters (body shape, bone length, joint angles, etc.), the corresponding model prediction parameters can first be generated and subsequently compared with the image-extracted features to calculate a single “error value” which can represent the degree of difference between the hypothesis and the observed values [10]. One study [11] projected the predicted 3D mesh onto a 2D image, adjusted to maximise the degree of overlap between the mesh and the capture target contours, and the iterative closest point (ICP) algorithm allowed for a comparison of the match between the visual shell of the image and each vertex of the capture target. The key to generative algorithms is the accurate definition of the algorithm function so that specific assumptions can be compared to image information; if the algorithm function is out of calibration, matching of optimal model parameters cannot be achieved, resulting in reduced motion constraints and increased probability of outliers [12]. Constructing highly robust algorithm functions for higher image noise and lower model configurations is more difficult, on the one hand, because generative algorithms require reasonably reliable initial speculation on model parameters and, on the other hand, the captured target needs to be initially calibrated in a specific pose at the beginning of the phase [13]. Without human intervention, the algorithmic function is not self-correcting and reversible due to degradation inaccuracy caused by occlusion, image noise, or other factors. Previous studies [14] attempted to improve the relevant algorithm functions or to address this difficulty by combining generative and discriminative algorithms.


## 3. Applying the Cam Shift Algorithm to Track Athletes

### 3.1. Colour Characteristics of Statistical Targets

Colour information is insensitive to translation, rotation, and deformation of the target and is therefore often used as a feature for target tracking. Among the many statistical methods used to characterise the colour distribution of an object, the histogram is one of the most commonly used. It is a parameter-free density estimator obtained by counting the number of pixels with the same pixel value in a region of interest. In order to calculate the colour histogram of a target object, the colour of the target is quantified into different classes so that pixels with similar colours can be clustered into the same histogram class while at the same time reducing the time and space complexity of the calculation process. The traditional histogram is calculated as follows: for a histogram withm rank values quu=1…m,(1)qu=∑x∈R∑y∈Rδcx,y−u.This is the target area for which the histogram is to be calculated, and functionℜ2⟶1,……,m is used to convert the pixel values with coordinates x,y to the corresponding histogram level values. δ is the Kronecker symbol.(2)δi−j=1,ifi=j,0,otherwise..In order to obtain a probability distribution within the range [0, 255], the histogram needs to be scaled. The following formula scales each level of the histogram from [0,qmax] to [0, 255]:(3)pu=min255qmaxqu,255,where u=1,…,mandqmax=maxquu=1,…m.


### 3.2. Calculating Probability Distribution Charts

The Cam Shift algorithm tracks a moving target by iteratively finding peak points in a probability distribution. A probability distribution is a special type of image in which each pixel value represents the probability that the pixel belongs to the target object. Any method that relates the value of a pixel to the probability that the pixel belongs to the target object can produce a probability distribution map, with the most commonly used method being the histogram back-projection [15].The histogram back-projection is calculated by first obtaining the colour information of the target object, representing it as a colour histogram, and then processing the input image as follows: for each pixel in the image, the pixel value is used as an index to find the histogram and replacing the pixel value with this statistic to obtain the probability distribution. It is usually necessary to scale the pixel values of the probability distribution so that the values are in the range [0, 255]. The steps to calculate the probability distribution are as follows:(1)
Convert the video image to HSV colour space and extract the values of the H component.(2)
Calculate the colour histogram of the target object.(3)
Calculate the probability distribution map. Let the value of the pixel located at (xy) in the video image be f(x, y) and P be the probability distribution map corresponding to this image. Then,(4)Px,y=pfIx,y,where puu=1…m is the colour histogram and function fIx,y finds the corresponding grade value in the histogram based on the value of Ix,y.


### 3.3. Applying the Cam Shift Algorithm to Track Targets

The Mean Shift algorithm can only be used for static probability distributions as the search window remains constant during the iterative process, whereas the Cam Shift algorithm can dynamically update the size and position of the search window during the iterative process of using the Mean Shift algorithm, thus being able to handle dynamic probability distributions [16]. The Cam Shift algorithm first selects a suitable colour space and then calculates the colour information of the target in that colour space. As the colour information of the target changes due to deformation, partial occlusion, and shadow interference during the target's motion, the Cam Shift algorithm needs to recalculate the colour information of the target in each frame and generate the corresponding probability distribution map when tracking the target [17]. In order to dynamically adapt to changes in the probability distribution caused by target motion, the Cam Shift algorithm applies spatial moments to adjust the size and position of the search window during the iterative process, which is the main difference from the traditional Mean Shift algorithm. The main steps of the Cam Shift algorithm are as follows [18]:(1)
Select the initial position of the search window.(2)
Apply the Mean Shift algorithm and save the results of the calculation (the centre of mass position and the zero-order moment of the window).(3)
Reset the position and size of the search window according to the saved zero-order moments.(4)
Repeat steps 2 and 3 until the algorithm converges. The Cam Shift algorithm can dynamically set the position and size of the search window during operation. The initial centre of the search window for the next frame is the same as the centre of the current search window, and the size and orientation can be calculated as follows: let,Ix,y be the value of the pixel located at xandy in the probability distribution, and x and y take values throughout the search window.Calculating the second-order moments of the search window,(5)M20=∑x∑yx2Ix,y,M02=∑x∑yy2Ix,y,M11=∑x∑yxyIx,y.Assumptions:(6)a=M20M00−xc2,b=2M11M00−xcyc,c=M02M00−yc2,where xcyc is the window centre of mass; the length and width of the search window for the next frame are as follows:(7)l=2∗a+c+b2+a−c22,w=2∗a+c−b2−a−c22.The direction of movement of the search window can be calculated using the following formula:(8)θ=12tan−1ba−c.When using the above steps to track the target, the Cam Shift algorithm calculates the centroid and area of the window within the range of the search window and resets the position and size of the next search window according to the calculation results [19]. Therefore, in the subsequent tracking process, it is not necessary to recalculate the probability distribution map of the whole image, but we only need to calculate the centroid of the current search window as the central position; a probability distribution map of the area slightly larger than the search window can be used, which can greatly reduce the amount of calculation [20]. Therefore, when the Cam Shift algorithm is used to track athletes in sports videos, its steps can be improved to the following form [21]:(1)
In the first image, set the calculation area of the probability distribution to the whole image.(2)
Select the initial position and size of the Mean Shift search window. This window generally corresponds to the target object.(3)
Calculate the probability distribution for the current search area. The centre of this area is the same as the centre of the search window, and the area is slightly larger than the search window.(4)
Iterate through the probability distribution using the Mean Shift algorithm and calculate the mass-preserving zero-order moments of the probability distribution (the area of the distribution).(5)
After calculating the centre of mass and zero-order moments in step 4, reset the position and size of the search window. Move the centre of the search window to the centre of mass of the conservation and the area as a function of the zero-order moments. Go to step 3.The above way of calculating the probability distribution reduces the amount of computation and improves real-time tracking [22, 23].


## 3.1. Colour Characteristics of Statistical Targets

Colour information is insensitive to translation, rotation, and deformation of the target and is therefore often used as a feature for target tracking. Among the many statistical methods used to characterise the colour distribution of an object, the histogram is one of the most commonly used. It is a parameter-free density estimator obtained by counting the number of pixels with the same pixel value in a region of interest. In order to calculate the colour histogram of a target object, the colour of the target is quantified into different classes so that pixels with similar colours can be clustered into the same histogram class while at the same time reducing the time and space complexity of the calculation process. The traditional histogram is calculated as follows: for a histogram withm rank values quu=1…m,(1)qu=∑x∈R∑y∈Rδcx,y−u.This is the target area for which the histogram is to be calculated, and functionℜ2⟶1,……,m is used to convert the pixel values with coordinates x,y to the corresponding histogram level values. δ is the Kronecker symbol.(2)δi−j=1,ifi=j,0,otherwise..In order to obtain a probability distribution within the range [0, 255], the histogram needs to be scaled. The following formula scales each level of the histogram from [0,qmax] to [0, 255]:(3)pu=min255qmaxqu,255,where u=1,…,mandqmax=maxquu=1,…m.


## 3.2. Calculating Probability Distribution Charts

The Cam Shift algorithm tracks a moving target by iteratively finding peak points in a probability distribution. A probability distribution is a special type of image in which each pixel value represents the probability that the pixel belongs to the target object. Any method that relates the value of a pixel to the probability that the pixel belongs to the target object can produce a probability distribution map, with the most commonly used method being the histogram back-projection [15].The histogram back-projection is calculated by first obtaining the colour information of the target object, representing it as a colour histogram, and then processing the input image as follows: for each pixel in the image, the pixel value is used as an index to find the histogram and replacing the pixel value with this statistic to obtain the probability distribution. It is usually necessary to scale the pixel values of the probability distribution so that the values are in the range [0, 255]. The steps to calculate the probability distribution are as follows:(1)
Convert the video image to HSV colour space and extract the values of the H component.(2)
Calculate the colour histogram of the target object.(3)
Calculate the probability distribution map. Let the value of the pixel located at (xy) in the video image be f(x, y) and P be the probability distribution map corresponding to this image. Then,(4)Px,y=pfIx,y,where puu=1…m is the colour histogram and function fIx,y finds the corresponding grade value in the histogram based on the value of Ix,y.


## 3.3. Applying the Cam Shift Algorithm to Track Targets

The Mean Shift algorithm can only be used for static probability distributions as the search window remains constant during the iterative process, whereas the Cam Shift algorithm can dynamically update the size and position of the search window during the iterative process of using the Mean Shift algorithm, thus being able to handle dynamic probability distributions [16]. The Cam Shift algorithm first selects a suitable colour space and then calculates the colour information of the target in that colour space. As the colour information of the target changes due to deformation, partial occlusion, and shadow interference during the target's motion, the Cam Shift algorithm needs to recalculate the colour information of the target in each frame and generate the corresponding probability distribution map when tracking the target [17]. In order to dynamically adapt to changes in the probability distribution caused by target motion, the Cam Shift algorithm applies spatial moments to adjust the size and position of the search window during the iterative process, which is the main difference from the traditional Mean Shift algorithm. The main steps of the Cam Shift algorithm are as follows [18]:(1)
Select the initial position of the search window.(2)
Apply the Mean Shift algorithm and save the results of the calculation (the centre of mass position and the zero-order moment of the window).(3)
Reset the position and size of the search window according to the saved zero-order moments.(4)
Repeat steps 2 and 3 until the algorithm converges. The Cam Shift algorithm can dynamically set the position and size of the search window during operation. The initial centre of the search window for the next frame is the same as the centre of the current search window, and the size and orientation can be calculated as follows: let,Ix,y be the value of the pixel located at xandy in the probability distribution, and x and y take values throughout the search window.Calculating the second-order moments of the search window,(5)M20=∑x∑yx2Ix,y,M02=∑x∑yy2Ix,y,M11=∑x∑yxyIx,y.Assumptions:(6)a=M20M00−xc2,b=2M11M00−xcyc,c=M02M00−yc2,where xcyc is the window centre of mass; the length and width of the search window for the next frame are as follows:(7)l=2∗a+c+b2+a−c22,w=2∗a+c−b2−a−c22.The direction of movement of the search window can be calculated using the following formula:(8)θ=12tan−1ba−c.When using the above steps to track the target, the Cam Shift algorithm calculates the centroid and area of the window within the range of the search window and resets the position and size of the next search window according to the calculation results [19]. Therefore, in the subsequent tracking process, it is not necessary to recalculate the probability distribution map of the whole image, but we only need to calculate the centroid of the current search window as the central position; a probability distribution map of the area slightly larger than the search window can be used, which can greatly reduce the amount of calculation [20]. Therefore, when the Cam Shift algorithm is used to track athletes in sports videos, its steps can be improved to the following form [21]:(1)
In the first image, set the calculation area of the probability distribution to the whole image.(2)
Select the initial position and size of the Mean Shift search window. This window generally corresponds to the target object.(3)
Calculate the probability distribution for the current search area. The centre of this area is the same as the centre of the search window, and the area is slightly larger than the search window.(4)
Iterate through the probability distribution using the Mean Shift algorithm and calculate the mass-preserving zero-order moments of the probability distribution (the area of the distribution).(5)
After calculating the centre of mass and zero-order moments in step 4, reset the position and size of the search window. Move the centre of the search window to the centre of mass of the conservation and the area as a function of the zero-order moments. Go to step 3.The above way of calculating the probability distribution reduces the amount of computation and improves real-time tracking [22, 23].


## 4. Experimental Results

The experimental data for the analysis of this system were obtained from an NBA game video with a size of 20 × 240. The video was divided into two groups: group A and group B. The colour distribution of the players in group A was not similar to the background, while the colour distribution of the players in group B was more similar to the background [24, 25]. Both videos were captured with a still camera. The athlete detection effect of the system is shown in Figure 1; the upper half of the figure is the detection effect of group A video, and the lower half is the detection effect of group B video. The athlete tracking effect is shown in Figure 2; the upper half of the figure is the video tracking effect of group A, and the lower half is the video tracking effect of group B.Figure 1
Effectiveness of system athlete detection.Figure 2
Systematic athlete tracking effect.In order to better analyse the algorithm proposed in this paper, the formula proposed in [26] is used to measure the efficiency of the algorithm.1(9)Precision rate=correct athlete area detectedtrackedall athlete areas detectedtracked.Based on the experimental results, the data in Table1 were obtained.Table 1
Experimental data statistics.
Experimental videoCount framesAthlete detection accuracy (%)Precision of traditional Cam Shift algorithm (%)Precision of improved Cam Shift algorithm (%)Group A80 frames98.69294Group B80 frames97.56286The experimental results show that the improved athlete detection and tracking algorithm in this paper has higher robustness in practical applications. The improved tracking algorithm also has better real-time performance, with a processing speed of 20 frames/second during the experiments.Compared to the Microsoft Kinect camera based on red, green, and blue (Red-Green-Blue, RGB) depth images, deep learning algorithms have fewer constraints on the distance between the camera and the target to be measured and the sampling frequency of the video recording. Current methods based on deep learning have been implemented to automatically estimate the centre of a human joint from a 2D RGB image and output the 2D coordinates in the image. Different tracking effects are shown in Figure3. Also, by using multiple cameras in conjunction, synchronising the 2D joint positions of the human body in multiview camera images and combining them with deep learning algorithms enables the localisation of human joint centroids and key bony marker points in 3D spaces. Deep learning computer vision research based on 3D human pose recognition is attempting to use an algorithm for pose position estimation and tracking, and studies have explored different tracking errors for 3D human pose recognition based on monocular cameras as shown in Figure 4.Figure 3
Different tracking effects.Figure 4
Different tracking errors.Based on the above research status, this paper reveals the potential applications of markerless motion capture in the field of motion detection and feature motion recognition, such as the interference-free recognition and fast feedback of athletes' movements in daily training competitions, which can provide reference for coaches' training decisions.


## 5. Conclusions

In this paper, the Cam Shift algorithm is improved to perceive the colour characteristics of the image processing based on the view system mechanism, i.e., converting the video image from RGB colour space to HSV colour space, using the H (hue) component to model the target object. Experiments in this paper show that the improved athlete detection and tracking algorithm has higher robustness in practical applications. The improved tracking algorithm also has better real-time performance, with a processing speed of 20 frames/second during the experiments.



---

*Source: 9401001-2022-04-18.xml*