From: dstewart+@cs.cmu.edu (David B Stewart)
Subject: Equalizing Tournament Scores (was Re: I'll be at PABA) 
Organization: The Robotics Institute, Carnegie Mellon University
Date: Tue, 26 Jan 1993 06:29:11 GMT

In article <1993Jan25.172817.5189@portal.vpharm.com> dap@portal.vpharm.com (David A. Pearlman) writes:
>In article <b1k3xqg@rpi.edu> sigma@degas.ipl.rpi.edu (Kevin Martin) writes:
>>sbb@panix.com (Steve Baumgarten) writes:
>>>It's not perfect, but how else could you do it?  I thought about
>>>coming up with an "average" score for each game, then the percentage
>>>you beat this average score would be multiplied by some set value for
>>>your total score.  Something like that.  But it's pretty hard to do,
>>>given all the different scoring opportunities on modern games
>>>(especially Williams games).  What's the "average" score for DR. WHO?
>
>A simple scheme that would seem to make some sense would be to:
>
>1) Determine the average score on each machine, Ai.
>2) Determine the standard deviation of scores on each machine, Ri.
>
>The scores on a second machine Sj  could then be made roughly equivalent to
>those on a machine i, Si, as:
>
>Sj (modified) = Ai + (Ri/Rj) (Sj-Aj).
>
>This would be much fairer than a scheme where high scores (either average
>or potential as evidenced by a high standard deviation) on some machines
>nullified the importance of other machines.

Although I can't make it to the tournament, here are my 2 cents, just
in case down the line I do make it to a tournament.

I like the idea of making the scores more fair across machines, mainly because
the current format gives an unfair advantage to those people who are better at
the highly inflated machines (e.g. Cue-Ball Wizard).  If I were to participate,
I would feel disadvantaged because I don't have access to a CBW to practice.

However, the problem I see with the above statistical method is that you don't
know who wins until everyone has qualified.  Two people may be fighting for 1st
and 2nd place.  Both have played, and have scores X and Y, which are quite
close to each other.   The winner will be decided by how *other* people in the
tournament perform, because the average and derivation continually change.
Therefore at 5:45pm X might be in 1st place, at 5:50pm Y goes into
first place, and at 5:55pm X is back in 1st place.

Here is a proposal, which doesn't have the problems of the above statistical
method, but does make each machine in the tournament equal value, immune to any
kind of point inflation.  With this format, you can have both Fun House
and Cue-Ball Wizard in the same tournament, without making Fun House scores
meaningless!

    - For each game in the tournament, give a standard set of points
	to the top 50 players (or top 100 ... this is variable) on each
	machine.  For example, 1st place gets 200 points, 2nd place 175, 3rd
	place 150, ... 10th place gets 75 ... 25th place gets 25, ... 49th
	place gets 2 points, and 50th place gets 1 point.
	    (obviously the exact point values for each point position can be
	    modified to better suit the tournament).

    - A player's overall score is then a summation of the points they 
	accumulated on each of the 8 or 10 machines in the tournament.  No
	matter what kind of point inflation a machine has, the grand champion
	on that machine gets a fixed number of points.  With 10 machines in the
	tournament, the maximum score is 2000 points if that person becomes
	grand champion in every game.  (or if you like point inflation,
	multiply everything by 10, and they get 20000 points :-) :-))

    - The top 16 players, based on the ranking of the overall scores, make the 
	finals.  If the final round is played on multiple machines, then the
	same format as for the qualifications can be used there too.

Since the tournament has 8 or 10 machines especially to test a person's skill
on a variety of machines, it eliminates the possibility that the person that
can play CBW, and only CBW, gets 10 Billion points on that game alone, and only
1 million on each other game for a total of 10,009,000,000 points, and still
finishes first in the tournament.  Instead, they get 200 points for that game,
which put them equal with the person who scored 3_billion on Fish Tales, but
behind someone who is capable of scoring 3B on both FT *and* TAF! 

This format is a little more difficult to maintain scores as compared to
just adding a players score; but I think it is better than relying on
any kind of statistical analysis method, as it does not take into 
consideration the performance of the not-so-good players.  Only the
good scores are taken into consideration.  This format, however, is
fairly easy to administer if you have a personal computer on hand,
that is properly programmed (pay me a trip to PAPA and I'll program
it for you :-) :-) ).   You can make a new printout every 15 or 30 minutes,
and post that printout for everyone to see the most up-to-date scores.

I'm sure that there are a few loose ends with this format which should
be written in stone, but I think it would be a lot more fair then
last years (and this year's proposed) format.

~dave

-------------------------------------------------------------------------------
David B. Stewart  - email: <dstewart@cmu.edu>            The Robotics Institute
snail mail:       - ECE Dept., Carnegie Mellon University, Pittsburgh, PA 15213
Current Projects: - Chimera 3.0 Real-Time Operating System
		  - Reconfigurable Sensor-Based Control Systems