From: langsl@hhcs.gov.au
Newsgroups: aus.games.roleplay
Subject: Re: Judging Role-Play
Date: 29 Sep 92 18:06:42 +1000
Organization: Aust. Dept. Health, Housing and Community Services

In article <s903609.717638722@minyos.xx.rmit.OZ.AU>, s903609@minyos.xx.rmit.oz.au (ACE) writes:
> 	  However, this is beyond the ORIGIONAL question

  So?  The topic of conversation has obviously moved on.  And considering
  the amount of response, some people at least found it interesting.

  But to get back to the original question...

> HOW do you judge roleplaying. Do you use scoresheets, word of mouth, player 
> voting, etc.

  To Matt (& interested others) - if you still want some suggestions...

A METHOD - no claim is made that this is the best or only...use as you will.
=============================================================================
- Score sheets can be good.  They provide a standard format for all judges to
  use.  The effort of filling one in forces some degree of evaluation at the 
  time of the player's roleplaying, so you don't have to only rely on memory 
  a day or so later.  A well written score sheet, clearly identifying what 
  categories to score on, and how to score (ie score as number 1..5, as given
  below) can make the process a _LOT_ easier on the poor ref. 

- Scoring?  Its going to be subjective, based on each judge's opinion and
  experience.  Experience with judging, and with each other's points of view 
  will hopefully help judge's scores to be more consistent with each other.  

  You might try to practise by having 1 ref, some players, and extra judges as 
  observers - judge independently, then discuss with each other, then discuss
  with the players to get feedback both ways.  If you have the time and 
  people - you might be horrified/amazed (or both) at the results.  

- What are you scoring on? What aspects of roleplaying do you and your fellow
  ref's feel are important?  What do you think is important?  What will be 
  important in the scenario?  If you can define this then you can break down
  the judging  into those defined categories - otherwise just have a single
  score.

  A Contrived Example:  
  --------------------
  1)Characters are defined as being Free Trader starship crew, successful so far
  through being hardworkers, and supporting each other.

  So you could judge players on how 'success oriented' they played the
  character, and how well they helped each other out.  A player whose charac-
  ter is successful at the expense of the others rates high on one(4.0)
  and low (2.0) on the other category, and overall would not be any better
  than someone who was satisfactory (3.0 and 3.0) - except that since the 
  second player satisfactorily played the Free Trader stereo type he/she 
  'passes' and rates better than the first player who 'failed' to play to type.

  And this is only one way of manipulating the numbers. 
                  
- To make it easy for the judge, try using simple ratings such as 
 
  poor           = 1 out of 5, or an 'E' rating
  unsatisfactory = 2                 'D'
  satisfactory   = 3                 'C' 
  good           = 4                 'B'
  excellent      = 5                 'A'

  Someone who is better than good, but not excellent, is thus a 4.5, or B+.

- And leave room on the score sheet for comments.

- Tip - get the players to fill in the sheet with their names and character
  names in the order they are sitting - this saves you time, and effort, and
  makes it easier to remember who was whom based on where you remember they
  were sitting.

- After you rate all players in the team individually, rate them in order -
  who did you think was best, second best, third, etc.  How does this ranking
  compare with your scores?  If it doesn't then work out why.  These two
  approaches help your consistency.  The ranking helps you work out best in 
  the team, but the absolute grading then helps compare teams.

  Do your scores ASAP after the game.

  When you have done a second team as above, compare it with the first - who
  was best, second best, third,...etc.  And then check how consistent your
  scores are.

  Try to make sure different ref's see the same team - comparing their scores
  helps to work out what their numbers mean - someone might consistently score
  people as 3s that someone scores as 3.5s.

  ****NOTE: the numbers are a _PROP_...!  Use common sense and judgement, talk
  with the other refs and thrash it out as you like - but something similar 
 to the above may make judging the sessions quicker and easier, and the 
 choosing of prize winners less traumatic -- at least I hope it does.


Alistair Langsford.


Newsgroups: aus.games.roleplay