Predicting success in introductory psychology from early testing: High consistency and low trajectory in multiple-choice test performance

The Introductory Psychology course is the flagship of the undergraduate psychology curriculum and often functions both as a core curriculum course for non-majors and as a requirement for psychology majors. As a result, Introductory Psychology classes frequently have significantly larger enrollments than other psychology courses and are usually taught as lecture courses (Silverstein, 1982). Students enrolled in large classes of Introductory Psychology are also usually evaluated and graded based in part, if not entirely, on their performance on multiple-choice tests. Psychologists as educators have been comfortable with the multiple-choice evaluation format because of its advantageous properties: high reliability, ease of administration, and minimal bias in scoring (Handleman, 1977). A well-constructed multiple-choice test can probe knowledge and understanding over a wide range of content areas, even in a field as conceptually challenging as chemistry (Barnett-Foster & Nagy, 1996). The ability of multiple-choice items to test a wide range of content is ideal for assessing learning in Introductory Psychology, a course that surveys topics that are very diverse (Griggs & Jackson, 1996). The use of multiple-choice tests has not been without controversy. On one hand, some investigators assert that multiple-choice tests are only capable of indexing superficial, fact-oriented learning (Williams & Clark, 2004). But research has also shown that a well-constructed multiple-choice test can in fact assess critical thinking and the understanding of abstract concepts (Wallace & Williams, 2003). Interestingly, psychologists have also documented significant correlations between performance on multiple-choice tests in psychology courses and various proxies for general intelligence, such as standardized college admission tests scores (Bleske-Rechek, Zeug & Webb, 2007; Griggs & Jackson, 1988). Thus multiple-choice tests in a college survey course may not only assess how much a student has learned, but also the student's fundamental ability to learn. We investigated another property of multiple-choice testing: the high consistency of multiple-choice test performance over the course of a semester. This performance consistency has been evident from high positive inter-correlations across successive multiple-choice tests in Introductory Psychology (Griggs & Jackson, 1988; Thompson & Zamboanga, 2003). We first replicated prior work on inter-test correlation on successive multiple-choice tests in Introductory Psychology. We were particularly interested in examining the extent to which percent performance on the first multiple-choice test in Introductory Psychology is predictive of percent average performance on three subsequent multiple-choice tests in the course. We predicted that the correlation of Test 1 performance vs Post-Test 1 performance would yield values of Pearson's correlation coefficient that are as high or higher than any particular inter-test correlation. In other words, we expected that Test 1 would prove to be a stronger predictor of subsequent multiple-choice performance. We also investigated whether the score on a single unannounced ("pop") 10-item multiple-choice practice quiz prior to the first multiple-choice test is a significant predictor of overall multiple-choice test performance in the course. In addition, in a post hoc analysis, we used hierarchical linear modeling to calculate performance trajectory over multiple-choice tests in Introductory Psychology. Two additional post hoc analyses explored (1) the issue of whether low vs. high scoring students drive performance consistency, and (2) the issue of whether performance on tests of "experimental psychology" was significantly correlated with performance on tests of "social/clinical psychology." METHOD Participants The participants were 458 students at a private university. …