From: bis@chaos.cs.brandeis.edu (Bis)
Subject: Statistical Rating Systems
Organization: Bis Information Services
Date: Fri, 11 Dec 1992 19:12:50 GMT

In article <1g08irINN8t5@master.cs.rose-hulman.edu> erwinaw@NeXTwork.Rose-Hulman.Edu (Anthony W Erwin) writes:
>In article <1992Dec5.235138.19179@news.cs.brandeis.edu>  
>bis@chaos.cs.brandeis.edu (Bis) writes:
>> Can there possibly be a worse rating system??  No offense, but this is
>> a completely meaningless, arbitrary rating system.  It gives a huge
>> advantage to those who are given more minutes of playing time, it
>> arbitrarily assigns greater weight to abilities that happen to be
>> represented by larger number on the stat sheet.  These are just two
>> concrete things that are wrong with the rating system, but on the
>> whole essentially what you're doing is throwing a whole bunch of
>> unrelated numbers into a heap, and assuming that they'll somehow
>> magically work themselves out to give you a good idea of who's better
>> than who.
>> 
>> In fact, any number-crunching rating system is only marginally
>> meaningful, because it has to make arbitraty decisions about what's
>> more important than what, ignores abilities and liabilities that
>> aren't recorded by statistics (such as hogging the ball, making
>> good/bad decisions, being a good/poor team player, keeping other
>> players from getting close to the basket, wearing other players out)
>> and doesn't consider the player's role on the team (for example, the
>> go-to guy gets certain stats inflated, and good players playing in the
>> shaddow of or as backups to big stars get some statistics deflated).
>	
>	Admittedly, this is not a perfect rating system.  You made a good point
>in saying that no rating system is totally correct because they fail to measure  
>certain things that are not reflected in statistics.
>	Although not perfect, I do believe that this system has some merit.   
>You say, "Can there possibly be a worse rating system?"  I would like someone
>to name a better one.  This system is not "completely meaningless."  I've known  
>coaches who use similar systems to measure their players performances.  The  
>only thing that they did differently was to put added weight on certain  
>categories that they felt were important.  For example, steals may be  

This is of no minor importance.  As I said, any rating system has to
make some arbitrary decisions about how to weight the different
categories, which is a matter of opinion and not objective fact.  But
your system doesn't do that at all, at just takes the importance of
any given category to be proportional to whatever kinds of numbers the
ability happens to put on the stats sheet.

>multiplied by two.  I feel that, at least with the data that I had to work  
>with, this is among the best rating systems.  It is certainly betterthan just
>looking at scoring average for example.  My system was not just a bunch of  
>numbers thrown together, but it was positive statistics minus negative  
>statisitics.  A higher rating indicates more made shots, more rebounds, more  
>steals, more blocks, and more assists than someone with a lower rating.  This
>rating system incorporates all the major statistical categories.  Another  

That's what I mean by a bunch of numbers thrown into a heap--you take
a bunch of statistics, add the ones that are considered positive,
subtract the ones that are considered negative, divide by the number
of games played, and assume that that will tell you who'd better than
who.  It just doesn't.

>argument for my system is that the players one would expect at the top of the
>list are there.  One does not see Will Perdue in the top five for instance.

Okay, now the issue is, what's the point of a rating system?  To
refrain from contradicting the obvious, or to give you information?  I
didn't say that your rating system widely skews the results and rates
some of the worst players as the best and the best players as the
worst.  Obviously, if player A is fifty positions above player B, you
can assume that player A is better than player B.  However, for any
specific player A and player B that would be more than 50 apart in
your rating system, you'd know which is better without the rating
system.  My objection is that the rating system doesn't tell you
anything you don't already know.  Did you think Will Perdue was one of
the best players in the league?  Of course he's among the worst, and
you don't need the rating system to tell you that.  What this rating
system can't do is tell you if Robinson is better than Jordan, or how
either of those compare to Barkley, or if Tony Massenburg is better
than Will Perdue.  

To summarize my point, a staistics-crunching rating system will tell
you what you don't need a rating system to know, and can't tell you
what you would want a rating system to reveal.

>	I do believe that the stats that I included do measure at least  
>partially some of the tings that you say it does not.  For instance, assists  
>are included.  An assist usually indicates that a good decision was made and  
>that the player is acting as a team player.

It only patially reflects one very specific kind of good decision.
I'm not simply talking about making a good pass to someone who
immediately takes a shot (and even then, an assist is only credited if
the shot falls).  Here's an example: a player, let's call him Barry
Lird, has an opportunity to shoot an uncontested 20-foot J.  But Barry
sees another player that has on open lane to drvie to the basket.
Instead of taking the jumper, he passes to the other player, who
drives to the basket and gets a dunk or a layup.  Barry's decision is
a very good one that increases the chances of getting a basket, but it
doesn't show up in the statistics (it's not an assist since the other
player has to do some maneuvering of his own to get the hoop).

(This is a digression that belongs in other threads, but I would
contend that these kinds of decisions are something Larry Bird
excelled at, making his whole team better without getting credit for
it on the stats sheet.)

>	I don't know how much of an advantage players with more playing time  
>have in this system.  The more time you're in the more opportunities you have  
>to get turnovers, foul, and miss shots.

Again, here is where the random weighting is a liability.  Again
you're assuming that your system just somehow works out to account for
things you don't really know it accounts for.

Try your system for 87/88, and I'm certain Reggie Lewis will show up
near the bottom of the list, simply because K.C. Jones didn't like to
play rookies.

>                                         I have included a list where instead  
>of dividing by games, I divided by minutes.

What bothers me here is the word "instead".  Why does the criticism
that your system doesn't account for minutes imply that you don't need
to account for number of games played?  The two are no more mutually
exclusive than steals and blocks.  If your original system had omitted
steals and someone pointed that out, would you then put steals in
*instead* of blocks?

>                                             The list is similar except for the  
>fact that many players are much farther up the list then they should be.   
>BARDO, S is far from the best player in the NBA.  He is number one when the  
>ratings are divided by minutes because he probably played for a couple of  
>minutes during the year and ended up with a couple of rebounds or something.   
Case in point.

>Playing more minutes should be rewarded in some way anyway.

Refer to what I said above about Reggie Lewis.  You're putting too
much faith in coaching decisions, which can be wrong.  Furthermore, it
has a lot to with who someone plays with.  Darryl Dawkins plays far
fewer minutes than he used to because now he plays with Jeff Hornacek.
Does this mean he's gotten worse?  This also goes for stars--if they
have good backup players for the postion, they play fewer minutes.
You could argue that they're getting more rest, which improves their
performance when they are on the floor, but then you'd once again be
assuming that all these complicated factors just somehow work
themselves out and account for each other, when you don't really have
a reason to assume so.

>	Does anyone else have an opinion on this or other ratings systems?   
>Again, I want to say that there are several flaws with this one, but I think  
>that it is just about as good as any statistical rating system.

Okay, one thing needs to be clarified:  I'm criticizing statistical
rating systems in general, and I'm just saying that this one is worse
than most in that you don't do any weighting and in that in one case
you don't account for minutes and in another you don't account for
games played.  

>
>(Ratings divided by minutes instead of games)
>NUMBER OF PLAYERS: 424
>AVERAGE RATING: 0.157
>      NAME          RATING
>----------------------------------
>BARDO,S             1.000

As with any rating system, either you give credit where it's not due,
or you have to pick out the anomalies by hand.

>ROBINSON,D          0.547
>LEONARD,G           0.538
>SCHEFFLER,S (*)     0.533
>LANE,J (*)          0.500
>DAUGHERTY,B         0.463
>RODMAN,D            0.463
>OLAJUWON,A          0.461
>BABIC,M             0.457
>MALONE,K            0.454
>STOCKTON,J          0.454
>WILLIS,K            0.437
>GRANT,HO            0.432
>NANCE,L             0.423
>BIRD,L              0.419

Wow, Bird is way up here even when you account for minutes
(considering that he played a lot of minutes) and only played half the
season and games aren't taken into account?  Sorry, Bird-bashers!  :)

>BARKLEY,C           0.418
>JORDAN,M            0.414
>SCHREMPF,D          0.402
>KEMP,S              0.393
>WILLIAMS,J          0.391
>EWING,P             0.391
>ELLISON,P           0.387
>JOHNSON,L           0.384

[etc...]