Assessment of Learning Potential: Construction and First Evaluation of the Psychometric Characteristics of an Analogical Reasoning Test
Authors: Berger Jean-Louis 
Published: July 2011

This master’s thesis (Berger, 2003) concerns a new learning potential test of analogical reasoning, the Hessels Analogical Reasoning Test (HART; Hessels, 2003) aimed at the assessment of pupils from 5 to 15 years of age in a group situation. A frequently emphasized problem of learning potential tests is the time needed for their administration. We intend to be able to assess a whole group of approximately 20 pupils in the context of their classroom, in a relatively short time of about 45 to 60 minutes.

The analogies are presented in two different formats: 2 rows x 3 lines with six response alternatives or 3x3 with eight response alternatives. The number of elements varies from one to three, as does the number of transformations. We created nine series of increasing complexity for a total of 70 items. The items were constructed by pairs, meaning that two items had the same number of elements, and the same number and kind of transformations applied. The complexity, that is, theoretical difficulty, was defined by the number of transformations and elements present in the analogy. For example, an item with one element and one transformation is easier than an item with three elements and two transformations. The procedure was divided into two phases. In the first phase, a collective introduction was offered using four example items aimed at familiarizing the pupils with the tasks and the different formats of the matrices. Immediately after, a pre-test combined with training (after each item an explanation was given about the transformations applied) was administered using the first set. The second phase was a static post-test administered a few days after the pre-test/training using the parallel forms of the pre-test/training items. For each degree, a series of items was defined, according to level of difficulty, varying between 12 (1st grade) and 20 items (6th grade) for each phase of the test.

We administered the HART to 117 pupils of a primary public school (mean age 8;11). In addition, these pupils took the Standard Progressive Matrices of Raven (SPM) and an arithmetical test in a static and collective administration. Teachers of each class completed a rating scale for each of his pupils about three noncognitive variables (participation in the lessons, application in schoolwork, and behavior in class) and two cognitive variables (school success in French and mathematics).

The results showed that the training caused great inter- and intraindividual variation, explained by the learning process taking place during this phase. Due to this variation, internal consistency was low for this phase. Thus, for subsequent analysis, we only considered the reliable results of the post-test. Of main interest were the correlations between the HART and the other variables measured. The noncognitive factors given by the teacher’s judgments showed lower correlations with the HART than with the SPM. For instance, the HART showed a correlation of .08 (ns) with pupil’s behavior, whereas the SPM showed a correlation of .21 (p<.05). This result means that the score offered by the HART is more independent of behavior in class. Moreover, the arithmetic test is more correlated with the learning test than with the SPM. Finally, a stepwise regression analysis demonstrated that the SPM predicted 14.2% (F1,116=19.151; p<.01) of the variance of success in mathematics; the HART predicted an extra 4% (F1,115=5.557; p<.05). For French, the stepwise regression analysis shows that the HART has a slightly superior predictive validity.

These first results show that the instrument can be used in a group situation and has promising properties. The research will be extended to different populations, with variations in the procedures and methods.