To prevent this, assessments are being developed such that teaching to the test will actually improve instruction. Such performance assessments, as they are called, have students work on complex, intrinsically valuable, authentic tasks. Olae is a performance assessment for Newtonian physics. It is based on student modeling, a technology developed for intelligent tutoring systems. Students solve traditional problems as well as tasks developed by cognitive psychologists for measuring expertise.
Students work on a computer, which records all their work as well as their answers.
This record is analyzed to form a model of the student s physics knowledge that accounts for the students actions. The model is fine-grained, in that it can report the probability of mastery of each of 290 pieces of physics knowledge. These features make Olae a rather unusual assessment instrument, so it is not immediately obvious how to evaluate it, because standard evaluations methods assume the assessment is a short-item test. In many cases, more data would be required in order to adequately access Olae, so this paper should be viewed more as a methodological contribution than as a definitive evaluation.