Qualifying sets are used to test the readers' ability to score accurately and to supplement the training provided by the training sets. These sets include responses from all activities to be scored, and they are formatted to resemble the portion of the Answer Book the team will score. Each team scores the open-ended student responses and assigns the appropriate score point on a customized scan sheet. At grades 3 and 5, where most teachers work across subject areas, teachers score various subject areas. Team Leaders, who assist Scoring Coordinators, receive three days of training.
Scorers receive two to three days of training.
Quality Control . As inconsistencies and inaccuracies are detected, Scoring Coordinators and Team Leaders hold discussions with the team and assist individual readers to improve accuracy. Accuracy sets are administered on two or three mornings each week to determine whether teams of readers maintain appropriate levels of accuracy during the scoring process. Readers who score below 70 percent on any accuracy set receive additional training immediately from the Scoring Coordinator or the Team Leader and are released from retraining only after the scoring problems are resolved. If scoring problems are not resolved, the reader is dismissed from the scoring project.