Evaluating Players & Performances Judiciously — T20
By: Amol Desai
Tags: Data Science, Cricket, Sport Analytics

The little bit of additional complexity isn’t in vain

Here is how this model performs compared to a naive model that uses the average runs scored per ball at each batting position to estimate expected runs per delivery. Each data point here is an average across match situations for a batting position, competition and venue. We can clearly see the benefit that additional contextual information brings.

Comparison w/ a more naive model

The survival probability model also works well. Here, it is shown in aggregate for different batting positions.

Model vs actual — survival probability

Still not quite there

Now, that we have context adjusted expected runs per delivery, we can calculate the average difference in RPO per batsman/bowler from the average batsman/bowler under the same context. However, the sample size issue rears its head again here. If we just use the raw output of the model at this stage, the top 5 batsmen based on RPO are as shown below.

Sorted model output — top 5

This is clearly not useful. We need to adjust our confidence in the relative RPO numbers based on the number of balls faced by a batsman. The more data we have for a particular player, the more confident we can be in the estimates. If a batsman has played fewer balls, we conservatively say that they are likely to be closer to the average and need more noteworthy performances in either direction before we can be confident that they are really better or worse than the average.

Accounting for sample size

And, voila!

After applying this model, batsmen with more evidence, for whose numbers we can associate higher confidence, bubble up in the top 5.

Final sorted model results — top 5

We use a similar approach to aggregate the probability of survival across multiple innings as well. There, the volume is in terms of number of innings instead of number of balls.

The Metrics

We have two (Edit: Three. I added Evidence Weighted Relative Wickets Per Over to this.) metrics that we have developed here:

Evidence Weighted Relative RPO (EWR-RPO): This is a measure of how many runs more or less a batsman scores in one over relative to the average batsman coming in at the same position, in the same competition, on a similar ground, under similar circumstances. For bowlers, this translates to runs conceded in one over relative to the average bowler bowling to batsmen coming in at the same position, in the same competition, on a similar ground, under similar circumstances. For batsmen, a higher value of EWR-RPO is better, while for bowlers a lower value is better. For bowlers, it might also make sense to multiply this by 4 to get a more intuitive measure, but I am not doing that here. 0 is a par value for both and represents the average bowler/batsman. Evidence Weighted Cumulative survival probability of average batsman or Evidence Weighted Survival Factor (EWSF): This is a measure of how likely it is for an average batsman coming in at the same position, in the same competition, on a similar ground and facing deliveries under similar circumstances to have gotten out by this time (the ball before he loses his wicket, if he gets out in that innings, or the last ball that he plays, for innings were he remains unbeaten). EWSF ranges from 0 to 1, although 1 is almost unattainable. A higher EWSF is indicative of the extent to which batsmen play longer than average innings. Using this for bowlers is tricky because although higher EWSF indicates a well set batsman, it also indicates a batsman who has played longer than others and hence, may be more likely to get out. I also didn’t see the value of using this for bowlers at this time, given that I plan to introduce match impact related metrics in a follow up.

Let’s take a look at what these metrics look like in tandem for batting:

Batting metrics

We are clearly able to see the value of some of the usual suspects here in the top right periphery of the chart, where players that play longer innings and score at a fast rate, reside. However there are some interesting insights that emerge as well. We see that some of the bigger names in openers that show up here play more of an anchoring role than a short blaze. This is somewhat in line with my findings on how teams that slowly ramp up without losing wickets in the power play, and that use this platform in the middle overs do better. L Ronchi wasn’t one of these openers. Munsey isn’t either. Both Ronchi & Munsey are ahead of their peer openers in their respective leagues in both scoring rate and longevity of innings, consistently. Another interesting player for me was Rahkeem Cornwall, who doesn’t quite play the long innings that some of the top openers play, but he consistently scores fast and has an above average time spent at the crease. His bowling EWR-RPO is at -0.135 which is only at the 30th percentile level. The top bowlers based on EWR-RPO are as follows:

Bottomless Usage Potential

The basic approach that I have described here can be extended in a variety of ways. Since our original model is built for expectations per delivery, we can build a whole set of these follow up models adjusting for confidence based on sample size. We can have additional models per batsman against bowler type (spin/pace), bowler against batsman type (LHB/RHB), player per competition , player per position etc. (even matchups, although I am a little more cautious there because of small sample sizes) The list is almost endless and depends on the analyses that we want this to power.

Naive averages are sufficient for use overall splits due to larger sample sizes

For some cases, where there are enough samples that can be aggregated for fairly confident estimates e.g. bowler type, simple aggregations can work.

But most cases where we are comparing players need the additional step to account for the sample sizes. The below table illustrates this using the case of evaluating batsmen’s RPO against spin.

Evidence weighting is needed at the player level & leveraging higher level signals is helpful

The evidence weighted estimates of Relative-RPO are more robust than the naive average. Moreover, if we use only the performance data of each batsmen against spin, we have fewer sample sizes and so our evidence weighted algorithm is less confident about attributing a larger Relative-RPO number. Using additional signal from the batsmen’s overall perf (for the same sample size, we can be more confident in a batsman’s numbers against spin if they have also done well against pace) lets us be confident about the naive average numbers for larger sample sizes while being pessimistic about the estimates with smaller sample sizes. Finally, adding the signal from overall performances against spin lets us reign in over-aggressive estimates a bit more by requiring more evidence for a batsman’s score to be farther away from the overall Relative-RPO for spin.

No free lunch

Before parting, let’s briefly acknowledge a couple of loopholes that we have left or created here and how I plan to address them.

I’ve introduced metrics to address how fast batsmen score runs, the longevity of their stay and bowler economy rates here. However, we haven’t arrived at a similar evidence weighted relative metric for measuring the wicket taking ability of a bowler (Edit: I have, since writing this, added EWR-WPO which is the evidence weighted relative wickets per over metric for bowlers. The relative aspect of this comes from a comparison against the probability of a wicket falling in a given scenario. This probability is based on the above explained scenario/context only, and doesn’t take into account ball outcome history). While relative RPO (emphasis on relative) for batsmen and bowlers generally shouldn’t be a bad proxy for impact on the match result, wickets taken or lost in different contexts have different impact on the match result. I’ve shown before how boundary rate in different phases of the game impacts match outcomes. It is likely that relative RPO may not provide the complete picture of player contribution towards the result of a game. To address these, I will be working on a complementary match impact model to evaluate players based on how outcomes influence the outcome of matches.

Secondly, by taking into account available evidence for the metrics here, we’ve made it hard to evaluate players for whom we have less evidence because of smaller sample sizes. This could apply to new players, but also to players who may have played fewer deliveries in a particular context e.g. against left arm bowlers, across competitions, in a specific position etc. Moreover, since our metrics evaluate relative to the expectations in a given context, players performances get evaluated for that specific context. This doesn’t say much directly about how a player would fare in other contexts. A player like Munsey who has done exceedingly well in associate T20Is shows up on the top, but there is still a question mark on how he would fare in the IPL.

For these cases, my approach would be to find other similar players in the context where a given player has a non-trivial sample size of evidence and who have played a non-trivial sample size of evidence in the desired context, and use their metrics to do a projection for the player of interest. For e.g. if player X has played in the CPL & PSL, but not in the IPL and we want to evaluate how this player might do in the IPL, I’d take players from the CPL & PSL that have similar numbers to player X within some threshold of similarity. Of these players Y1, Y2…Yn, I’d take those who’ve also played in the IPL and take a similarity weighted average of how they have done in the IPL to estimate how player X might do. A similar approach could also be applied in time to evaluate how new players would fare in upcoming years.

p.s. I specifically use “he” here because the data that I used was from men’s T20s. The approach applies to women’s cricket as well and I will be prioritizing looking at women’s data in the future.