‘Football Weather’-Diving Into The Effects of Weather on NFL QB Performance
By: Josh Mancuso
Tags: NFL, Pandas, Data Science, Python3, Statistics

As an avid fan of the NFL and perhaps an even bigger fan of fantasy football, one source of frustration that has always plagued me in my fantasy football hobby has stemmed from my uncertainty about how much of an effect an inclement weather forecast will have on the game and the statistical production of the players in it. I have often found myself overreacting to a forecast of rain and wind, benching my QB only to miss out on a big game in spite of the weather. Was this just small sample/bad luck, or was that ‘bad’ weather in actuality not a meaningful detriment for a professional caliber athlete? If 10 mph winds and light rain is no big deal, what about snow? How about 15 mph winds? 20? What about temperature? How extreme in either direction does the temperature need to be before it starts to have a meaningful impact? These are the questions I set out to answer, or at least shed some light on, in this blog post.

It was surprisingly difficult to find historical weather data for NFL games, but fortunately with some fairly simple python code, I was able to scrape some basic weather info for almost every game between 1985 and 2016 from profootballarchives.com. I also was able to find some NFL statistical data, although unfortunately it was not exactly in game by game box score format. From Kaggle NFL Datasets, I decided to use the QB game log data. With a little wrangling, I was able merge the datasets together in a pandas DataFrame.

After quite a bit of cleaning, renaming columns, categorizing variables, binning some continuous data, and converting/recasting datatypes, the dataframe was ready to dive in for some analysis. First up, since quarterbacks get paid to pass the football through the air, I figured that looking at data for different wind speeds would be a good place to start. I binned the data into 4 categories: 0–10 mph, 10–15, 15–20, and 20+. Let’s take a look at the wind effects on several measures of QB performance:

As the wind speed category increases from left to right for each subplot, there’s a pretty clear downward trend in QB performance. Production metrics (yards, TD passes, and fantasy points), as well as efficiency metrics (yards per attempt, QB Rating, and completion percentage) all suffer as the game average wind speed increases.

Next, I analyzed the affects of precipitation, broken down into 5 categories: None (or indoors/Dome), light rain, light snow, rain/storm, and snow/storm.

Strangely, it appears that light rain has a more profound negative affect than rain/storm. However, the error bars for light snow, rain/storm, and snow are quite large, which indicates that perhaps this data is not reliable. I ran some statistical tests for significance, and the resulting p-values indeed bear that out, with p-values of 0.75 for light snow, 0.26 for rain/storm, and 0.09 for snow. This just means that the results could be due to random variance or confounding variables (such as wind speed/temperature- more on this later).

It appears that temperature, on its own, doesn’t seem to have much of an effect. While it seems reasonable to assume that extremely hot or cold temperatures would have a real-world effect on quarterback performance, particularly extreme cold, once again the large error bars for those categories indicate insufficient sample sizes for statistical significance. Just for fun, I decided to chart a heatmap of the NFL teams who play the most games in the coldest temperatures.

Neat- OK so we’ve looked at how the isolated weather categories affect QB stats, but in the real world, these weather categories rarely occur in isolation. What is the impact from games that have wind AND precipitation, across a range of varying temperatures? Using plotly express, in just one line of code we can visualize all these variables together:

There’s a lot going on here: each dot or bubble represents one QB game played. The temperature is represented by color, the number of fantasy points scored corresponds with the height on the y-axis, and the wind speed is represented on the x-axis. Additionally, the size of the scatterpoint represents the maximum wind gust for that game, which is why the larger bubbles mostly appear on the right side of each graph in games where the avg wind speed is higher. Finally, each subplot represents the precipitation category for that game. The two major takeaways from this graph are the negative correlation between fantasy points and wind speed/wind gust, across all precipitation categories, as well as the general downward trend and lower overall fantasy points scored in games with precipitation.

The plot above shows the same data (minus temperature), but has all the games grouped together, and here the color of the bubbles represent the precipitation category.

Now that we have seen some graphical representation of the overall effects of wind, temperature, and different types of precipitation, I really wanted to dive in to analyze QB performance in games played in great weather vs bad weather. To qualify as great weather, it had to be pretty pristine, where no element of the weather should have an impact on QB performance. I took a subset of the overall data where the temperature was moderate (between 60 and 80 degrees), clear skies (or in a Dome) with no precipitation, and an average wind speed of under 5 mph. For bad weather, I took a subset with rain/storm or snow (light snow and light rain were excluded), avg wind speed above 12 mph, or extreme polar temperatures (below 10 degrees or above 95 degrees). The resulting subsets each comprised a little over 2000 QB games played, a decent sample size. Lets look at how much bad weather impacted QB performance:

Negative % difference for all categories indicates worse QB stats in bad vs great weather, as expected

The main takeaway here is that production metrics generally have a larger percentage decrease in bad weather. Efficiency metrics, while also generally worse, aren’t as pronounced. This is likely due to coaches calling fewer pass attempts in bad weather games. Perhaps in the future I will take a look at running back data and look for a corresponding increase in volume/rush attempts in bad weather.

Finally, I wanted to look at a few individual players and compare how they perform in great vs bad weather games. I chose 3 players from teams who play in home fields which frequently have bad weather, Tom Brady of the New England Patriots, Ben Roethlisberger of the Pittsburgh Steelers, and Aaron Rodgers from the Green Bay Packers. To represent teams whose home game are played in a Dome/good weather, I chose Peyton Manning from the Indianapolis Colts, and Kurt Warner from St Louis Rams / Arizona Cardinals.

Unsurprisingly, all 5 QBs averaged more fantasy points in great weather vs bad. The bad weather acclimated QBs each score between 2 and 3 more fantasy points in great weather games. Also unsurprising is the drastic negative downtick for Kurt Warner. While the modest difference for Peyton Manning was a bit of a surprise at first glance, Manning was well known for his preparation and a meticulous work ethic, and it shows in the consistency of his performance, regardless of any challenges from adverse weather.

Lastly, as a lifelong die hard New Orleans Saints fan, I would be remiss if I didn’t take a peek at the stats for Drew Brees. It is fairly widely claimed in the fantasy football world that Drew Brees has some of the more drastic home-road splits of anyone in the league, meaning that his performance drops off significantly when he leaves the friendly ‘Dome-field advantage’ of the Mercedes Benz Superdome. Is this characterization accurate? Lets see if the numbers back up that reputation:

Say it ain’t so, Drew!!

Alas, it pains me to admit, the pundits are right..

In summary, here are the major takeaways I gleaned from exploring the NFL weather data as it relates to QB performance:

The effect of wind/wind gusts is the most significant weather variable affecting QB performance. The effects are fairly slight for avg wind speeds from 0 mph up to about 15 mph, but above that threshold, the difference in fantasy points is about 12% less. In games with winds over 20 mph, the negative effects are more significant, with ~17% fewer fantasy points per game.

Precipitation, on its own, is basically inconclusive. Although the average fantasy points are lowest for games with light rain, the data for games with rain/storm and snow are not statistically significant.

Temperature, as an isolated weather variable, doesn’t have much of an affect on QB performance.

QBs across the board tend to score 8–10% more fantasy points in great weather than in bad weather. A small handpicked sample of QBs who play their home games in a dome or great weather cities, on a relative percentage basis, didn’t perform much worse in bad weather than QBs who tend to play in bad weather more frequently.

Thanks for reading! For a deeper dive into the data and analysis behind this blog post, here is the link to my github repo.