You start to look for the one that's funny, original, for the kid who takes a different tack. But graders must adhere to the scoring guide, no matter how creative the answer. The process includes safeguards to increase the likelihood that two scorers will give the same paper the same grade. Trainees practice by scoring a set of pre-graded papers, then discussing differences between their grades and the grades of more experienced scorers. To qualify to work on actual tests, they must score another set of pre-graded papers and hit the target score a minimum number of times.
Trainees get three chances to qualify or they're out.
About 10 percent get cut. Even once they are in, their scoring is constantly checked. A pregraded item may be slipped among other items they are grading to see if they come up with the same score. Trainers monitor the accuracy of each scorer by randomly checking one in every 10 scored papers. If there is a disagreement, they discuss it.