From: nolan@helios.unl.edu (Michael Nolan)
Subject: Re: CJA Awards
Reply-To: nolan@helios.unl.edu
Organization: University of Nebraska - Lincoln
Date: 26 Aug 92 04:08:29 GMT

cfc@isr.harvard.edu (Christopher F. Chabris) writes:

>I'm not sure I qualify as a "strong player" by Mark's definition, my peak
>rating being 2320 and my current rating being 2267 (2270 FIDE), but I was one
>of the three judges who handled the awards. 

I'm curious as to what Mark thinks is a 'strong' player', too.  [Probably
NOT the majority of Chess Chow subscribers.]  I wonder what he thinks about the 
fact that the typical editor of a chess publication in the USA is NOT an IM 
or GM, and these are the people making the decisions as to what to publish 
in the first place.  (Chess Chow and Inside Chess excepted.)

>Each judge was asked to rate all
>of the entries in each category ("Best Analysis" in this case) on a 5-point
>scale. The Chief Judge then compiled these ratings to determine the winner
>and Honorable Mentions in each category. He will have to explain the procedure
>he used to do this, if anyone asks, but I assume it was just a simple average
>of the three judges' ratings. 

The judges were told to grade on a 1-5 scale, in quarter-point intervals,
with a 5 being awarded only to exceptionally outstanding articles.  The
total score of the winning articles was well over 11 for virtually EVERY
winning article, and generally over 12.  There was one article that got
a score of 14 1/2 this year, last year there were a couple of 15's.
(I do not release scores of individual articles or judges, however, although
I told Chris he was free to do so.)  What often happens is that each judge's 
TOP choice is low rated by one other judge, and so the 2nd choice of each 
judge gets the award.  If anybody has a suggestion for a better way to select 
the winners in 1993, I'm ALL EARS!!  Judging awards such as these is by
its very nature a subjective process.  Long detailed judging criteria would 
result in the criteria of the chief judge (or whoever writes them) as a 
substitute for the good intentions and HARD WORK put in by the judges.

>This was a very time-consuming task, as there
>were well over 100 total entries in about 25 categories, for which we were
>paid nothing but a one-year CJA membership that would have cost $10. To have
>better, or perhaps higher-rated judges, would probably cost money. This could
>be financed by an entry fee, which might also cut down on the number of entries
>(some categories had several entries from the same publication). But that is an
>issue for next time, when I will no longer be involved.

I wish that CJA had more money with which to reward the judges for their
effort.

>Perhaps the judging can be improved next year by funding it and hiring a wide
>variety of people with more expertise to spend more time on each entry. This
>year, I did the best I could with the limited time available and tried to be
>as objective as possible. 

FWIW, there is a $1 per item charge for nominating, which is used to defray
the postage costs of shipping the items to the judges, and the cost to
have the award certificates printed and mailed.   ($30 or more PER JUDGE to
ship by overnight courier, and it always is a last-minute thing requiring
the speedy delivery!)  This charge was added last year, and does
not seem to be a material factor in the number of entries, and I am
considering raising it to $2 or so for 1993, with the possibility of
offering the judges a 'small' honoraria.  A $5 or more charge would probably 
hurt the small publications.  (I don't think it would even dent the budget 
of Chess Life or Chess Horizons.)

Every year I've been involved in the CJA awards (3 years as a judge, 3 as 
chief judge) there have been charges of poor judging, biased judging, etc.  All
I can do is repeat my earlier comment to Mark (and any others) that I would
be delighted if they would volunteer to judge in 1993.  Then somebody else
can claim that THEY are biased!!

Chris, Hanon, Ira, and the judges from the past two years all have my 
deep and heartfelt thanks for doing what Mark has demonstrated is a 
thankless task.

Mike Nolan
Newsgroups: rec.games.chess,rec.games.board