A Model for Assessing Complex Educational Outcomes.

EDRS Pric! MF-$0.65 HC-$3.29 Classroom Observation Techniques, *Curriculum Evaluation, Discover2rocesses, *Educational Objectives, *Evaluation Teehnignes, Formative Evaluation, Measurement Techniques, *Models, Student Evaluation, Summative Evalccion, *Test Construction, Test Reliability, Test Validity A general model was built for assessing important zompiex educational goals. Literature on t.erformance tests, process goals, and verbal protocols were critically examined. Advantages of these methodologies were incorporated into the model which was tested in an individualized mathematics inquiry laboratory which used whole-task approach. Five process goals were chosen to represent mathematics inquiry. Task analyses and observations determined criterion behaviors for each. Criteria were outlined for test items administered to children who were also observed in cl,%ss. Additional validation studies are needed but some behaviors of tlis educationally important complex goal appeareu to be tapped. Appendices include the detailed and categorized process goal of analysis and planning for the task, the problem situation it _,elE, scoring key, a sample verbal protocol, the coding scheme used in observing the child's approach to the task, and a classification of strategies used. (Author/TA) A MODEL FOR ASSESSING COMPLEX EDUCATIONAL OUTCOMES M. Kathryn Meese University of Pittsburgh INTRODUCTION U S. DEPA RTMENT OF HEALTH, EDUCATION I WELFARE OFFICE OF THIS DOCUMENT HAS SEEN EDUCATION REPRODUCED EXACTLY AS RECEIVED FROM THE PERSON OR ORGANIZATION ORIGINATING IT. POINTS OF VIEW OA OPINIONS STATED DO NOT NECES, SAPILY REPRESENT OFFICIAL OFFICE OF EDU CATION POSITION OR POLICY After three years of research, the Commission on Tests appointed by the College Entrance Examination Board (1970) has reccmmended that the Scholastic Aptitude Tests of verbal and mathematical ability be eliminated as tools for making decisions about college entrance. In their place, the Commission proposed a "flexible assortment" of tests of complex educational outcomes, such as commitment to social responsibilities, sensitivity, adaptability in novel situations, and artistic talent. This dissatisfaction with tests that sample relatively simple kinds of learning has been a recurrent theme in education, but few tests of these complex kinds of learning have materialized. As a result, we find educators paying homage to these complex variables, while assessing them impressionistically or not at all, since appropriate evaluation instruments and procedures are unavailable. Time and time again, we find that those involved in innovative educational programs report dramatic changes in their schools while the data from standardized tests of skill learnings reveal no significant differences between experimental and control groups. This suggests that either educational programs are having no impact or that tests are not detecting the changes; in many cases the latter conclusion seems more probable. Much of the evidence we now have about complex educational goals has come from factor analytic studies such as those of Thurstone, Cattell, and Guilford. Unfortunately, it is not clear how this information can be used to develop measures to detect the ways in which learning CYD of such complex goals is influenced Ly instruction. Because those who have studied outcomes such as creatisrity, problem solving, and thinking C\/ have failed to analyze them into behavioral ce7.ponents, we do not know which aspects of behavior should be selected for evaluating instructional programs designed to promote these kitds of goals. In discussing this issu-,, Glaser (1967) has cautioned that "if, indeed, complex reasoning and open-endedness are desirable aspects of human behavior, then this CID needs to be a recognized and measurable goal. Overly general, nonperformance based objectives may force us to settle for what can be easily expressed and measured [p.2]". (1111: In this paper I shall first present a general model for developing instruments to assess students' progress in curricula designed to promote complex educational outcomes. I shall then describe an application of the model to a test designed to assess outcomes of an Em4 individualized mathematics inquiry lab. Briefly I shall consider some of the ways in which the development of such a test affect? d the curriculum work.