Helping AI understand team sports with BigQuery and GPS data
By: Paddy Green
Tags: AI, Bigquery, Google Cloud, Sports, Gps

In this article I’ll walk you through the SQL statements that I used to preprocess GPS sport data for machine learning. The aim was to train a model to make predictions on future games. In order to help the model understand as much about the game as possible I decided to generate features that could generalise across the whole dataset.

BigQuery is a data warehouse in Google Cloud that offers a free tier for investigative projects. It has multiple built in GIS functions for geographical data processing and provides machine learning capabilities through SQL. These attributes make it an ideal place to load GPS sport data. See link below for how to load data into BigQuery.

Averaging Coordinates

After my GPS data was loaded I converted my latitude and longitude fields into a coordinate field using the SQL below.

ST_GEOGPOINT(sensor_long, sensor_lat)

As the granularity of my GPS data was very fine (1 coordinate /team/millisecond), I decided to average the coordinates to save on storage and query costs. I did this by using the ARRAY_AGG and ST_CENTROID function and saved the results into a permanent table like this:

SELECT timestamp, ST_CENTROID_AGG(location)

FROM `<PROJECT>.<DATASET>.gps_data`

GROUP BY 1

Filtering Coordinates

To further remove surplus data I filtered out coordinates that did not reside on the pitch.

Using google maps https://www.google.com/maps/d/u/0/ I drew a Polygon for each pitch and added two LineStrings to the sides of each pitch that I wanted to use as axes. I then loaded both geography files into BigQuery by converting from KML to geoJSON by adding the geoJSON to a column in a CSV file. An example data format is shown below.

# pitch.csv polygon;name

{"type": "Polygon", "coordinates": [[[x,y], ...]]};Pitch1

# leftside.csv linestring;name

{"type": "LineString", "coordinates": [[x,y], ...]};Pitch1 ....

Using my pitch data in BigQuery I filtered the data to leave only the coordinates on the pitch using the following SQL.

WHERE ST_CONTAINS(

ST_GEOGFROMGEOJSON(polygon),

ST_GEOGPOINT(sensor_long, sensor_lat)

) = true

Projecting Coordinates

The next transformation I made was to project the coordinates to a distance_from_left, distance_from_top coordinate system that generalises across the entire dataset.

Using the LineString geographies drawn in google maps I calculated the distance at right angles to the side of the pitch to every coordinate with the following SQL statement. This gave me a new coordinate system for my data where the origin is always the top left of the pitch.

ST_DISTANCE(ST_GEOGFROMGEOJSON(left_side_of_pitch),coordinate) AS

distance_from_left,

ST_DISTANCE(ST_GEOGFROMGEOJSON(top_side_of_pitch),coordinate) AS

distance_from_top,

Feature Generation

Using my new coordinates system I engineered some new features that were relevant to the understanding of the game. For example the distance between adjacent players.

To calculate the adjacent player I used analytic function which allows me to use an aggregate function on a window of the data. Ordering by distance_from_left I grouped the list of players with an ARRAY_AGG and partitioned by second and team. I then used an UNNEST and the OFFSET field to fetch the coordinate of the player to the left of the current coordinate from the the grouped list of players. This query is shown below.

WITH collect_player_list AS (

SELECT

sensor_game_seconds,

sensor_player_id,

player_loc,

ARRAY_AGG(player_loc) OVER (

PARTITION BY sensor_game_seconds, sensor_team_name

ORDER BY distance_from_left

) AS players

FROM `<PROJECT>.<DATASET>.grouped_gps_data`

WHERE player_loc IS NOT NULL

), player_position_from_the_left AS ( SELECT *

FROM collect_player_list,

UNNEST(players) AS closest_player WITH OFFSET position

WHERE ST_EQUALS(closest_player, player_loc)





) SELECT

sensor_game_seconds,

sensor_player_id,

player_loc,

CASE

WHEN position > 0 THEN players[Offset(position-1)]

ELSE NULL

END AS player_on_the_left_loc

FROM player_position_from_the_left

Combining Data Sources

I combined the GPS data with some manually labelled event data using a timestamp in order to generate a column for who is in possession of the ball. Due to the sparseness of the event data I needed to fill in the NULL possession values for every row of GPS data between events.

To achieve this, I used one analytic function to keep track of the number of possession changes so far in the game. I then used a second analytic function partitioned by player and the number of possession changes to fill in the NULL possession values. This query is shown below.

WITH event_and_gps_data AS ( SELECT *,

CASE

WHEN possession_flag THEN 1 ELSE 0 END AS in_possession,

FROM `<PROJECT>.<DATASET>.event_and_gps_data`





), running_sum_of_change_in_possession AS ( SELECT *,

SUM(in_possession) OVER (

PARTITION BY 1

ORDER BY sensor_game_seconds

) AS change_in_possession_running_sum

FROM event_and_gps_data





)



SELECT *,

CAST(

MAX(in_possession) OVER (

PARTITION BY sensor_player_id,change_in_possession_running_sum

ORDER BY sensor_game_seconds

)

AS BOOL) AS interpolated_possession_flag

FROM running_sum_of_change_in_possession

Data format for modeling

The final thing I needed to generalise across the dataset was the individual players. This was important as it meant that the model could understand the role of each type of player across the entire dataset.

As I knew the position of every player, I generated separate columns for each individual position and feature using the shortened SQL below. This allowed my model to understand the types of players and not just their behaviour on the pitch.

SELECT sensor_game_seconds,

ANY_VALUE(IF(positionName='LWinger', speed, NULL)) AS lwinger_speed,

ANY_VALUE(IF(positionName='LWinger', x, NULL)) AS lwinger_x,

ANY_VALUE(IF(positionName='LWinger', y, NULL)) AS llwinger_y, ANY_VALUE(IF(positionName='Halfback',speed, NULL)) AS halfback_speed

... FROM `<PROJECT>.<DATASET>.preprocesses_data`

GROUP by sensor_game_seconds

To Conclude

I generated unbiased training data that allowed my model to understand the game better and make predictions on future games.