ترجم ما يلى الى العربية:"see Ketterlin-Geller, Shivraj, Basaraba, & Yovanoff, 2019, for an 
approach to analyzing whether the selection of particular distractors is consistent with the 
ordering predicted by the LP). 
Several researchers have pointed out that person classification is not a straightforward 
process (or may even have potential pitfalls) and that the level of a student’s performance is to be 
expected to fluctuate across performances, depending on both the demands of the task and the 
student’s perception of what is required. For example, Battista (2011) wrote, “Research on 
learning suggests that quite often, the state of student learning is not neatly characterized as 
‘being at a specified level,’” (p. 542), and Corcoran et al. (2009) observed that “At any given 
time, an individual may display thinking/practices characteristic of different points on the path, 
due to features of both the assessment context and the individual’s cognition" (p. 39). Daro et al. 
(2011, p. 24) argued that classifying students into LP levels is “a matter of making probabilistic 
judgments,” and we agree—a person classification should at least be accompanied by a measure 
of certainty in the classification. For example, when using IRT methods, an alternative to 
providing a person classification is to produce a vector of probabilities that the person is at each 
level. 
Item/response classification is not a straightforward process either. Confrey et al. (2019, 
p. 29) expressed that “we do not expect that all items at one level of an LT will be less difficult 
than items at the next level.” They referred to this expected variation as “intra-level” variation 
and provided several examples of sources of intra-level variation, such as number type (Table 2, 
p. 29). For example, an item variant that uses fractions may be more difficult than a similar 
variant that uses whole numbers. 
10 
In spite of these challenges, we would argue that both item/response classification and 
person classifications have the potential to be helpful, if provisional judgments based on an LP 
are to guide a student’s learning. As outlined in the Attali and Arieli-Attali framework, the 
validity and reliability of these classifications can be explicitly examined, given ideal design. An 
ideal design includes multiple instances of items/responses that address each level, scoring of 
items/responses by multiple raters, and multiple measures of learning. An ideal report expresses 
the degree of certainty around classifications. In practice, it is unlikely that all these ideals can be 
met in a single study, but attention to them can help researchers both recognize limitations of a 
given study and direct subsequent research efforts.
Possible Revisions to the LP Validation Cycle 
During a meeting conceptualized by Leanne Ketterlin-Geller and Jere Confrey that 
explored the application of LPs to classroom assessment, we discussed the Graf and van Rijn 
(2016) LP validation cycle and how it might be further specified. In addition, revisions to the 
cycle were influenced by the Attali and Arieli-Attali (2019) validation framework. A number of 
these proposed revisions are discussed here. 
Suggested Revisions: Theory Development 
It was pointed out that design experiments, which involve ‘“engineering” particular forms 
of learning and systematically studying those forms of learning within the context defined by the 
means of supporting them (Cobb, Confrey, diSessa, Lehrer, & Schauble, 2003, p. 9), could be 
used to inform theory development. Most often, theory development is based on the existing 
research literature, but sometimes the existing research literature on a particular topic is sparse, 
or even non-existent. In this case one could still apply a design experiment to the purpose, b