Models in environmental planning selection of impact variables and estimation of impacts

2 Parameter Selection and Collective Estimation of Interactions between Impact Variables in Models Concerning Environmental Planning . 2 Experimental Context . 5 Experiment 1: Repeated Measurement Analysis of Students Individual Estimates Concerning Expert and Student Selected Impact Variables . 7 Overview . 7 Method . 7 Results . 9 Experiment 2 and 3: Group Estimates Concerning Impacts of Environmental Variables . 10 Method . 10 Results . 13 Discussion . 19 Experiment 1: Experts in Environmental Planning . 19 Experiment 2 + 3: Collective Judgment on Variable Impacts . 20 References . 22 1 Research was carried out at Eidgenössische Technische Hochschule Zürich (ETH) and at AlbertLudwigs-Universität Freiburg im Breisgau. We thank Bertrand Lisbach, Petra Kielchling and Marc Schärli for their assistance in conducting and analyzing the 1998 group experiment. Additionally, we thank Marcus Cheetham and Debbie Johnson for their assistance in correcting the english manuscript. * Eidgenössische Technische Hochschule Zürich (ETH), Switzerland ** AlbertLudwigs-Universität Freiburg i.Br. , Germany Ralph Hansmann, Harald A. Mieg, Helmut Crott, and Roland W. Scholz 2 April 2002 Abstract This paper includes an experiment concerning the selection of impact variables in environmental planning and two uniform group experiments concerning the quality of group estimates of impacts. Participants were students of environmental sciences at ETH Zurich. Experiment 1 revealed that during participation in an environmental case study, students’ individual estimates of impacts of expert selected variables increased, as compared to the estimates of impacts of variables selected by students. Further impacts were estimated in two group experiments. The quality of the estimates was analyzed referring to expert estimates. Individual estimates at the end of group discussions and group estimates were more accurate than prediscussion estimates. However, within group means of the individual prediscussion estimates were as accurate as group estimates. A higher accuracy was obtained by the mean group estimates aggregated over all groups. key words: parameter selection, collective estimation, group accuracy, expert judgments. judgment bootstrapping Parameter Selection and Collective Estimation of Interactions between Impact Variables in Models Concerning Environmental Planning Environmental planning involves the problem of estimating the impacts of certain variables on other variables (Faludi, 1987; Selman, 1992). This problem is encountered in planning local sustainable urban development as well as in global climate change policies. For instance, traffic and industrial production influence the concentration of CO 2 in the atmosphere. In this case, we have three potential impact variables , (a) traffic facilities, (b) industrial production, and (c) atmospheric CO 2 concentration. The problem consists in assessing the strength of mutual impacts of these variables, for instance the impact of traffic facilities on industrial production or the impact of production on the atmospheric CO 2 concentration. The estimation of the impact that variables have on other variables involves two steps; the definition and selection of relevant variables and, thereafter, the estimation and aggregation with regard to the interaction between the variables. In order to define relevant impact factors we often have to refer to experts (Mieg, 2001). In psychology, there are two lines of research on experts and expertise, respectively. One line focuses on experts’ cognitive skills, in particular on their ability to define and recognize relevant problems. From this perspective experts’ performance has been found to be on a high level: experts have excellent diagnostic capabilities (Chase & Simon, 1973; Chi, Glaser & Farr, 1988; deGroot, 1965). The other line of research focuses on the judgment and decision making of experts. In the 1950s, Meehl and others compared clinical and statistical prediction, finding statistical models superior to the doctors’ predictions (Meehl, 1954). Subsequently, a phenomenon called judgment bootstrapping was described: Regression models using the experts’ predictions correlate closer with the criterion than the expert judgments the models are based on (Blattberg Models in Environmental Planning: Selection of Impact Variables and Estimation of Impacts ETH-UNS Working Paper 32 3 & Hoch, 1990; Camerer, 1981; Dawes & Corrigan, 1974). Consistent with this finding, Hogarth and Makridakis (1987), who reviewed the literature on expert prediction, recommended computed aggregations of expert judgments. However, expert performance depends on the field of expertise. According to Shanteau (1992), we have to divide the problem areas into two classes: One where the performance of expert judgments is consistently high, e.g., in meteorology, and one where it is low, e.g., management. Environmental planning combines expertise from both classes. Mieg (1993) argued that the bootstrapping effect is due to the reliability of automated aggregation of judgments. However, the definition of the relevant variables to be computed remains the task of experts. Experts are indispensable in identifying impact variables (first step) but, in some cases, could be substituted when it comes to aggregation (second step). The flexibility of human expert cognition is necessary to adapt to changing conditions (Winograd & Flores, 1986), in particular when being faced with ill-defined problems (cf. Scholz, Flückiger et al., 1997). The present study was conducted in the context of an environmental case study at the Chair of Environmental Sciences Natural and Social Science Interface (UNS) at the Swiss Federal Institute of Technology (ETH) Zurich. In UNS case studies, the definition and selection of impact variables for the construction of models of environmental systems, is usually done by graduate students. Thus the question arises: What would be the improvements, if some variables were selected and defined by experts? This question will be adressed in Experiment 1 which concerns the process of parameter selection. On a local level, environmental planning today often takes place in planning groups that combine experts, politicians and citizens. Particularly in Europe, we find round table discussions and so-called focus groups (Kasemir et al., 1997; Krueger 1988) in which decisions concerning environmental planning are made. In the Experiments 2 and 3 the processes of collective judgment in tasks concerning environmental planning will be analyzed. The two group experiments focus on whether, or to what extent, environmental judgments resulting from mathematically aggregating the judgments of different independent groups can be more appropriate than the judgments of single groups. A further interest of the group experiments was to find out, whether there was a systematic tendency in the direction of the opinion changes during the discussions. In psychological research, such shift phenomena (Lamm & Myers, 1978; Stoner, 1961, 1968) have been observed for a variety of discussion tasks (choice dilemma items, mock jury deliberation etc.). Normative influences and informative influences have been used as an explanation for such opinion shifts during discussions (Burnstein & Vinokur, 1977; Deutsch & Gerard, 1955; Goethals & Zanna, 1979; Stasser & Davis, 1981). Moreover, Social Decision Schemes (SDS; Davis, 1973, 1992) can also account for choice shifts with respect to group decisions. SDS can even be applied as an explanation for opinion shifts on the individual level, if the marked tendency of opinion convergence within groups (e.g., Crott & Werner, 1994; Godwin & Restle, 1974; Sherif, 1935; Stasser & Davis, 1981) and cognitive processes that tend to reduce cognitive dissonance (Festinger, 1957) are considered (see Crott, Zuber & Schermer, 1