Generic Problem Solving Skills: Are They Misnomers as Constructs of Educational Outcomes?.

Construct validity of a prototype baccalaureate generic problem solving skills test was explored through a structural analysis. The assessment method used three reallife problem situations crossed by six generic skills, namely DecisionMaking, Communication, Analysis, Synthesis, Valuing and Execution. There were six written responses for each skill for each problem situation. The structural model included 18 skill/task observations with 6 latent skill variables, 3 latent trait variables and a higher order general factor (g). A multitraitmultimethod design was employed using a confirmatory factor analysis (LISREL VI). The data set moderately conformed to the model with most of the variation in performance attributed to the general factor. Factor loadings were neither consistent across skills nor across tasks. Therefore, generic skills, as measured by this procedure, possess little independence from the general factor. The weaknesses in the assessment method and implications for test methodology and research in the assessment of generic problem solving skills are discussed. Generic Problem Solving Skills 3 Generic Problem Solving Skills: Are They Misnomers as Constructs of Educational Outcomes? Introduction While there has been considerable concern regarding the development and assessment of higher order cognitive processes (herein called generic problem solving skills) at all educational levels (NAEP, 1981; Bloom, 1984; Frederickson, 1984; Ennis, 1985; Sternberg & Baron, 1985), there is still controversy over the nature of such constructs and the structures of such abilities (Carroll, 1985; Heartel, 1985). More specifically, there is debate over whether such abilities are domain general or domain specific (Baron, 1982; Keil, 1982), or whether there is a hierarchical configuration including a higher order general factor and lower order specific factors (Gustafson, 1984; Carrol, 1985). Such issues converge at the confluence of cognitive science, psychometrics and education in which the first concerns their structure and development, the second their measurement, and the third the practice of helping individuals acquire them. This paper focuses on three perplexing issues undergirding the assessment of generic problem skills: 1) What are the structural relationships among such skills? ; 2) Are the skills generic across problem solving tasks or are they specific to individual tasks? ; and 3) To what degree are generic problem solving skills unique from one another or from a general factor? The answers to these questions may shed light on not only the construct validity of tests purporting to measure such entities, but also whether educational programs may Generic Problem Solving Skills 4 legitimately claim them as goals or desired outcomes. Generic problem solving skills may also be referred to as higher order mental processes (Bloom, 1984), critical thinking skills (Ennis, 1985), domain general skills (Keil, 1982), or reflective thinking skills (Dewey, 1933; Baron, 1981, 1982). They are considered as skills as opposed to capacities (Baron, 1982) since they are subject to the laws of learning (cue fidelity, reinforcement, practice, transfer) and can be acquired through educational or training experiences. Such skills are considered generic or ubiquitous in the sense they repeatedly occur in the analysis of skills underlying the resolution of real-life problems (Woditsh, 1977). Moreover, certain skills have been identified in a variety of contexts through a variety of 'research methods (Peterson & Watkins, 1 e79; Peterson, 1982). The most frequently occuring skills include Decision-Making, Communication, Analysis, Synthesis, Valuing and Execution. These skills, however, are subsumed under a general intelligence factor which represents the power of reasoning or the "noegensis" of abstract entities (Spearman, 1923), total life learnings (Carroll, 1982), or fluid intelligence (Horn and Cattell, 1966; Gustafson, 1984). One method used to assess generic problem solving skills is to measure them across a set of real-life problem situations analogous to the assessment center method (Moses & Byham, 1977). In this technique, an individual is presented with a real-life problem . ,. 4tuation (see Appendix I) and set of directive cues, one for each skill, designed to