Football players tracking — identifying players’ team based on their jersey colors using OpenCV | by C K | Towards Data Science

In my previous work, I used pre-trained Yolov3 model to detect and used SORT (simple online and realtime tracking) to track football players from video. Then I used OpenCV’s getPerspectiveTransform function to convert the video to bird’s-eye view.
towardsdatascience.com
One of the problems of this work is that the model cannot tell the difference between teams. It will be good if the program is able to identify players’ team instead of just detecting ‘person’. To further improve this, I wish to include a function which tells the difference based on the colors of players’ jersey.
Two approaches I can think of now to this problem.
Train an object detection model using custom dataset containing 3 classes — the players of two teams and referees. This approach might not be practical for real application because one has to train a specific model before every match.Use to current object detection model and extract the color information from the detections. Based on that I can identify the color of players’ jersey.
Train an object detection model using custom dataset containing 3 classes — the players of two teams and referees. This approach might not be practical for real application because one has to train a specific model before every match.
Use to current object detection model and extract the color information from the detections. Based on that I can identify the color of players’ jersey.
I decided to try approach 2 using OpenCV.
The stationary football video is downloaded from here.
“T. D’Orazio, M.Leo, N. Mosca, P.Spagnolo, P.L.Mazzeo A Semi-Automatic System for Ground Truth Generation of Soccer Video Sequences, 6th IEEE International Conference on Advanced Video and Signal Surveillance, Genoa, Italy September 2–4 2009”
The major difference among teams and referee is jersey color — one team is white, the other is blue and the referee is red. I want to determine the player’s team based on the jersey color.
First I identified the range for the three colors.
color_list=['red','blue','white']boundaries = [    ([17, 15, 75], [50, 56, 200]), #red    ([43, 31, 4], [250, 88, 50]), #blue    ([187,169,112],[255,255,255]) #white    ]
Then I applied OpenCV’s inRange function to create masks for three colors.
mask = cv2.inRange(image, lower, upper)
Then use bitwise_and to apply masks to the frame.
output = cv2.bitwise_and(image, image, mask = mask)
Three colors are extracted now
Now we need to apply these mask to the detection. I cropped the image based on the bounding box detected then applied the masks.
crop_img = frame[ymin:ymax, xmin:xmax]
The way I identify the color is to count the non-black pixels and calculated the ratio (non-black/total pixel) for the output images of the three colors. The highest ratio is the color of the jersey. For the example above, the blue has highest ratio, so the jersey is identified as blue.
Now I integrated this function to the player detection script, here are some examples of the masks and player detections.
Based on the results of jersey color, I can draw the bounding boxes with different colors.
Then similar to my previous post, by using Opencv’s getPerspectiveTransform, I obtained the bird’s-eye view as shown in the beginning.
There are still some problems:
Some players were not detected at some frames, and also some non-player people were detected. It might be improved by training an object detection model specifically for football players instead of using the pre-trained model. It would be interesting to also track the ball, even through the ball is really small, it is possible to try it or other object detection architecture.ID changed when there are player occlusion. Might be helpful to try other tracking algorithm like DeepSort.
Some players were not detected at some frames, and also some non-player people were detected. It might be improved by training an object detection model specifically for football players instead of using the pre-trained model. It would be interesting to also track the ball, even through the ball is really small, it is possible to try it or other object detection architecture.
ID changed when there are player occlusion. Might be helpful to try other tracking algorithm like DeepSort.
Some applications for this kind of tracking:
Track the players and calculate for their speed and distance.Get the heatmap for the players’ path for both teamsand more
Track the players and calculate for their speed and distance.
Get the heatmap for the players’ path for both teams
and more
Thanks for reading, suggestions and feedback are welcome.