From: glickman@husc10.harvard.edu (Mark Glickman)
Newsgroups: rec.games.chess
Subject: Re: Fischer-Spassky II
Date: 7 Sep 92 12:36:57 GMT
Organization: Harvard University Science Center

In article <1992Sep7.061723.28083@cs.brown.edu> jhc@iris.brown.edu.UUCP (James H. Coombs) writes:
>In article <1992Sep6.172221.11976@physics.ucla.edu> reynolds@physics.ucla.edu (Reynolds) writes:
>>[Performance ratings] are most accurate when the
>>score is about equal, then the players are about equal.  But when
>>the score becomes lopsided, the performance rating means very little.
>
>Evidence please.

Perhaps it would be clearer if Reynolds had said that when the score
becomes lopsided, the performance ratings are *poorly estimated*.
Observing a lopsided score between two opponents is drawing from
the "tails of the distribution", so that estimation is difficult.
To put in terms of an example, scoring 5-5 in a 10 game match is 
not that much different than 5.5-4.5, and the estimated performance
ratings will be close.  Scoring 9-1 is quite a bit different from 
9.5-0.5, on the other hand (the former says the first player performs
roughly 9 times better, and the latter says the first player
performs roughly 17 times better), so that the performance ratings
will be estimated to be quite different (as long as you don't use
the linear approximation calculation!).  But because 9.5-0.5 is
the best result one can obtain better than 9-1, this means there's
a lot of variability in the performance ratings calculated from these
lopsided scores; much more so than with the more evenly matched scores.

>>As has been stated, a perfect score against an opponent will not tell
>>you what your rating is - more data is needed - but the performance
>>rating formula gives an answer.  This answer is meaningless.
>
>The performance rating does not "give an answer" to the question of
>"what your rating is", if you mean by the latter, your cumulative
>rating.  It does not even attempt to provide a cumulative rating.  Was
>never intended for that purpose, etc.

As I understand it, the provisional rating formula is commonly used
to estimate performances, so that the original poster was using
"performance rating formula" in that context.  The performance
rating formula therefore *does* provide an estimate of your rating
from a collection of game results, but as the original poster notes
the estimate is meaningless when computed on a perfect result.

>>This is just my way of saying that the performance rating, while
>>couching itself in mathematics, is not a very worthwhile number.
>
>Well, if you look at the full statistical model, you will find that the
>performance rating is accompanied by measures of its expected accuracy:
>standard deviation and probable error.  Of course, accuracy is not
>"worth."

I'm not aware of any research that shows the error structure of
even the linear performance calculation as it pertains to predicting
game results.  Jim:  Evidence?

>If you read Elo, I think you will be impressed.

That is, you will be impressed at how much confusion Elo's so-called 
"derivation" of the statistical model can cause, and at the total disregard
of a sizable body of statistical literature on paired-comparison 
theory.

I don't recommend Elo's book for the theory he "develops."
I, for one, am not impressed.

            - Mark