Investigating the Effect of Domain Knowledge on Collaborative Problem Solving

Investigating the Effect of Domain Knowledge on Collaborative Problem Solving Timothy J. Nokes (tnokes@uiuc.edu) Michelle L. Meade (mlmeade@uiuc.edu) Dan G. Morrow (dgm@uiuc.edu) Elizabeth A. L. Stine-Morrow (eals@uiuc.edu) Beckman Institute University of Illinois at Urbana-Champaign Urbana, IL 61801 U.S.A. Collaboration and Expertise Previous work in cognitive science has identified a variety of factors that impact collaborative success and failure (see Kerr & Tindale, 2004 for a review). One hypothesis proposed to explain why groups sometimes fail to reach optimal performance is the notion of ‘production blocking’ (Diehl & Strobe, 1987). This is the idea that information produced by one member of the group may cause interference and disrupt the other members’ train of thought. We hypothesize that one way to reduce this interference is to have people who share expertise in the task domain collaborate. Prior research has shown that experts can remember more domain relevant information and are less susceptible to interference than novices (Ericsson & Kintsch, 1995). This suggests that experts may be better prepared than novices to benefit from collaboration. Experts’ domain knowledge may not only lead to a reduction in production blocking between group members but may also facilitate collaboration through the cross cuing of domain relevant information. To test these hypotheses we conducted a laboratory experiment in which expert and novice pilots solved flight related problem solving tasks either in collaboration with another participant of the same expertise level or alone. We predicted that the expert participants would show larger benefits from collaboration than novices. conditions, plane type, etc.). The narrative described a specific situation that occurred on take-off, en route, or on the approach phase of the flight. Each scenario had a simple and complex version with the complex version describing a problem with a less clear-cut solution (as determined by a separate sample of expert commercial pilots). Procedure. Participants read the four scenarios one at a time and for each identified the problem, generated possible solutions, and then selected the best one. Participants ‘talked aloud’ while solving the problems and wrote down their solutions on the problem sheet. Results and Discussion Table 1 presents the mean proportion of problems correctly solved by the expert and novice groups as a function of collaborative condition (collapsed across problem type). Table 1. Mean Proportion of Problems Solved and Standard Errors as a Function of Collaborative Condition. Individual Dyad Collaborative Effect Expert d = . 88 Inspection of the group means revealed a large effect of expertise showing that, as expected, experts solved more problems than novices (d = 1.14). There was also a large effect of the collaboration showing that participants working in dyads solved more problems than participants working alone (d = .62). Finally, the effect of collaborative improvement was larger for the experts than for the novices (d = .88 versus d = .46 respectively). These results suggest that experts may be better situated to benefit from collaboration than novices on domain relevant tasks. We are currently analyzing the verbal protocols to identify the social and cognitive processes that underlie this advantage. Methods Participants. Thirty experts (flight instructors with instrument ratings and commercial licenses) and twenty novices (first-year aviation students) from the Institute of Aviation at UIUC participated in the study. One pair of experts was excluded from the analysis because their performance was more than four standard deviations below the average performance for that group. Design. We used a 2 (novice vs. expert) X 2 (individual vs. dyad) X 2 (simple vs. complex scenario) mixed design. Expertise level and collaborative condition were between group factors and problem complexity was manipulated within groups. The experiment was composed of both a memory and problem solving phase, for the purposes of this paper we focus on problem solving. Materials. The problem solving tasks were texts of four flight scenarios that were each composed of a “set-up” and a narrative. The set up described the key dimensions of the problem situation (e.g., destination airport, weather Novice d = .46 References Diehl, M., & Stroebe, W. (1987). Productivity loss in brainstorming groups: Toward the solution of a riddle. Journal of Personality and Social Psychology, 53, 497- Ericsson, K. A. , & Kintsch, W. (1995). Long term working memory. Psychological Review, 102, 211-245. Kerr, N. L. , & Tindale, R. S. (2004). Group performance and decision making. Annual Review of Psychology, 55,