A New Expected Goal Model That is Better Than Corsi at Predicting Future Goals
By: Patrick Bacon
Tags: Analytics, Gradient Boosting, Machine Learning, Hockey, Data Analytics

A New Expected Goal Model That is Better Than Corsi at Predicting Future Goals

Alan Ryder broke ground in 2004 when he published hockey’s first expected goal model titled “Shot Quality,” but it wasn’t until Dawson Sprigings and Asmae Toumi published their expected goal model in October of 2015 that expected goals ascended into hockey popularity. Today, expected goals have usurped Corsi (shot attempts) as the go-to underlying metric for analyzing teams and skaters, and NHL arenas have even featured expected goal data on the jumbotron at intermissions. I recently published an article describing the false dichotomy between expected goal models and subjective shot quality analysis, the advantages of mathematically computed models, and the need for both.

One of the key benchmarks of an expected goal model is its predictive power. In fact, the predictive power of expected goals in general has recently been called into question in an article by an analyst known as DragLikePull. He compared the correlation between 5-on-5 score-adjusted Corsi/expected goal shares in the first half of the season and 5-on-5 goal shares in the second half of the season and found that Corsi was better overall at predicting second-half goals than expected goals were. Based on these findings, he concluded that fans and analysts should discard expected goals at the team level and return to using shot attempts.

My research has led me to different conclusions on the predictive power of expected goals, but before I get into that, I want to address my issue with this line of thinking. A part of me wishes that we had stuck to calling expected goal models “Shot Quality” models instead, because I think that the term “Expected Goals” implies that these models are solely predictive in nature, which isn’t necessarily the case. Even if expected goal shares were completely useless for predicting future goals at the team levels, expected goals would still be extremely useful for describing past events and telling us which teams relied heavily on goaltending and shooting prowess, or were weighed down by poor shooting and goaltending, and even which shots the goaltender deserved most of the blame for, so I disagree with the premise that hockey fans should stop using expected goals at the team level if they are not as predictive as Corsi.

Fortunately, I don’t need to fight tooth-and-nail to support my case here, because I have found an expected goal model that is more predictive of future goals than Corsi is: My own. I built an expected goal model that I will use as the driving engine behind a WAR model that I have built and will be releasing shortly. Before I get into the results of my tests, I will give a brief overview of how I built my model and the variables that I chose to account for. If you’re not interested in this and just want to see the comparison between expected goals and Corsi, simply skip to the test results.

I trained my model using extreme gradient boosting, a hyper-efficient machine learning technique commonly used for regression and binary classification problems. In other words, I showed my computer a bunch of shots, told it which of them were goals, and then used extremely powerful software to teach it to predict the outcome of new shots. I accounted for the following variables in my model:

Shot distance and shot angle. (The two most important variables.)

Shot type.

The type of event which occurred most recently, the location and distance of this event, how recently it occurred, which team the perpetrator was, and the speed at which distance changed since this event. (The inclusion of the last variable was inspired by Peter Tanner of Moneypuck.)

Whether the shooting team is at home.

Contextual variables such as the score, period, and seconds played in the game at the time the shot was taken.

Whether the shooter is shooting on their off-wing. (For example, a right-handed shooter shooting the puck from the left circle is shooting from the off-wing, and a left-handed shooter shooting from the same location is not.)

Additionally, I chose to make an adjustment for scorekeeper bias; an issue I’ve discussed in much greater depth in the past. The adjustment was quite rudimentary: for the past 3 seasons, I subtracted the average shot distance (by both teams) in all a team’s away games from the average shot distance in all their home games. The resulting value from this calculation was a scorekeeper bias adjustment factor which I subtracted from the reported distance of all shots; the difference from this calculation was “adjusted distance” and I used this for my model in place of reported shot distance. To give a quick example of this:

The average reported distance of shots taken in games at Xcel Energy over the past 3 seasons was 2.54 feet further from the net than the average reported distance of shots taken in games where the Minnesota Wild were the away team. This gives an adjustment number of 2.54 feet. If the reported distance of a shot taken Xcel Energy is 30 feet from the net, I subtract the adjustment number of 2.54 feet from the reported distance, and obtain an “adjusted distance” value of 27.46 feet from the net, which I use as the input for my model.

Most expected goal models are trained on a minimum of five full seasons of data to train and then “tested” on an out-of-sample season. I tried building a model using this approach, but I kept running into a major issue: My expected goal values never added up to actual goals. The total number of expected goals I calculated for a given season were typically somewhere between 100 and 300 below the number of actual goals scored in those seasons.

While the collective body of NHL shooters may perform better than the collective body of goaltenders over a given season, a massive discrepancy between the two of them persisting over multiple seasons suggests that expected goal values are too low. Indeed, my values were, and the reason for this is that the NHL shrunk reduced maximum goaltender pant sizes prior to the 2017–2018 season and reduced maximum pad sizes prior to the 2018–2019 season. These changes play a big role in the new high-scoring environment we’ve grown accustomed to over the past three seasons, and using an expected goal model built on old data prior to these changes holds goaltenders to an unfairly high standard and shooters to an unfairly low standard by underestimating goal probability.

In order to combat this issue, I chose to train my model on only the past three seasons. I began the training process using cross validation on the 2017–2018 through 2019–2020 seasons, testing the model on cross-validated samples using different parameters each time with the goal of finding parameters that would maximize area under the curve. The modeling technique which I used to train my model was similar to and inspired by the technique used by Evolvingwild in their expected goal model; the write-up for which was my introduction to the concept of extreme gradient boosting.

Unfortunately, I couldn’t just train the final model on every shot from the past 3 seasons and then test it at once, as this would lead to overfitting. (To put this more simply, if I showed my computer the shots that I was trying to test it on, it would become “too smart” and “cheat” in predicting goal probability by considering results it isn’t supposed to know.) In order to avoid overfitting, I removed 100 game samples from my training sample, trained my model on the other 3,524 games, and then “tested” my model on the 100 games in question, and repeated this process until I had tested my model on every game and saved the results. In total, this technically means that I built 74 different models for the past two seasons: 37 for even strength and 37 for the power play. But because each model was trained on almost all the same data, and each model used the exact same parameters and variables, it’s easier and still mostly accurate to say that I just built two models: one for even strength, and one for the power play.

I also built models for the 2013–2014 through 2016–2017 regular seasons using the same training and testing parameters as I used for the 2017–2018 through 2019–2020 seasons. In theory, I likely could have improved my results for these seasons by repeating the above process of cross validation for these seasons and acquiring different parameters, as these are different seasons with a larger total sample, but the main seasons that I care about are the past three, and any improvement from acquiring different parameters would be rather minimal, so I stuck with the same parameters. For these seasons, the gap between expected goals and actual goals was not a problem, so I just used all data from 2010–2011 through 2016–2017 and removed the test season from my training sample, building eight different models for the four seasons I cared about; four for even strength and four for the power play. I implemented the same scorekeeper bias adjustment for each of these seasons, but I used data from the three seasons prior to the test season in order to calculate a scorekeeper bias adjustment. I prefer this method intuitively and would have done this for the past three seasons as well, but scorekeeper bias has become a much smaller and more consistent (though still significant and problematic) issue in the past 3 seasons, and using older data would provide me with exaggerated adjustment values.

I spent some time working on an expected goal model for penalty shots and shootout shots, but I was remarked at how poorly they performed in testing. After some consideration, I decided to build the most rudimentary model possible for these shots, by assigning them all the same expected goal value of 0.31 goals, which reflects the percentage of shootout and penalty shot goals that became goals over the past three seasons. I am comfortable doing this because unlike all shots at other situations where external variables shape the opportunity available to shooters to score and goaltenders to save, variables such as the location, angle, and shot type in a shootout are all influenced almost exclusively by the shooter and the goaltender. (I didn’t bother to build an expected goal model for these shots prior to this season as they won’t factor into my analysis.)

I also worked on building expected goal models for shorthanded shots and shots taken with at least one goaltender missing, but I ultimately chose not to work extensively on these models or release the results because the sample size for these shots is quite small, and more importantly because my goal for this expected goal model was to empower a WAR model, and I did not plan to incorporate shorthanded offense or any plays created with an empty net into my model.