The tests were developed through a consensus process by international experts in mathematics, science, and educational measurement, and were endorsed by all participating countries. Working within the frameworks, test specifications were developed that included items representing a wide range of mathematics and science topics and eliciting a range of skills from the students. Each student in the sampled classrooms responded to one test booklet that included about 80 mathematics and science questions, requiring 90 minutes to complete. The items used in 1995 that were released for public use were replaced with items of similar content, format, and difficulty that were piloted in an extensive field test. About half the items used in 1999 will be released for public use.
Procedures used throughout the study ensure that the results are comparable across countries.
To ensure comparability in testing, rigorous procedures were designed specifically to translate the tests and numerous training sessions were held in data collection and scoring activities. Quality control monitors observed testing sessions and reported back to the International Study Center at Boston College. The samples of students selected for testing were scrutinized according to rigorous standards designed to prevent bias and ensure comparability. Prior to analysis, the data from each country were subjected to exhaustive checks for accuracy and consistency. About one-third of the items in 1999 were identical to those used in 1995, and the rest were quite similar in content, format, and difficulty.