Research settings in industrial and organizational psychology: Are findings in the field more generalizable than in the laboratory?

The authors analyzed for content all the empirical articles from the 1966, 1970, and 1974 volumes of the Journal of Applied Psychology, Organizational Behavior and Human Performance, and Personnel Psychology to determine the types of organizations, subjects, and dependent measures studied. Contrary to the common belief that field settings provide for more generalisation of research findings than laboratory settings do, field research appeared as narrow as laboratory research in the actors, settings, and behaviors sampled. Indeed, industrial-organisational psychology seems to be developing in the laboratory a psychology of the college student, and in the field, a psychology of the self-report of male, professional, technical, and managerial employees in productiveeconomic organizations. The authors suggest that coordinated strategies of research in both laboratory and field settings are needed to construct an externally valid industrial and organizational psychology. Should industrial and organizational psychologists conduct their research in laboratory settings? For many industrial and organizational psychologists, the answer to this question would be an emphatic no, but others would defend the laboratory as a legitimate setting for research (Fromkin & Streufert, 1976; Weick, 196S). The relative merits of laboratory and field settings have been debated within many areas of psychology, including comparative psychology (Miller, 1977), environmental psychology (Proshansky, 1976), social psychology (Ellsworth, 1977; McGuire, 1967; Ring, 1967), race relations (Fromkin & Ostrom, 1974), and perception (Gibson, 1966). Two common criticisms of laboratory research are that laboratory settings are susceptible to experimental artifacts (Rosenthai & Rosnow, 1969; Silverman, 1977) and . serious ethical problems (Kelman, 1967). Although these two criticisms have been debated in numerous articles and have not been resolved entirely, Fromkin and Streufert have argued persuasively that artifacts and ethical problems also exist in field settings and are not unique to the laboratory. A third criticism of laboratory research is that the artificiality of the laboratory imposes severe restraints on the external validity of findings. Even those critics who concede that laboratory settings provide for more elimination of alternative interpretations of results than the typical field settings do are often doubtful that one can generalize findings from the laboratory to predict, understand, or control real-world phenomena. A perusal of the recent Handbook of Industrial and Organizational Psychology (Dunnette, 1976) reveals the differences of opinion that exist among industrial and organizational psychologists with regard to the generalizability of laboratory research. Ohapanis (1976) observed that "most laboratory experiments in psychology have only very limited relevance for the solution of practical Vol. 34, No. 2, 141-150 Portions of this article were presented at the meeting of the Midwestern Psychological Association, Chicago, May 1977. The authors' names are listed in alphabetical order, reflecting equal contributions by both to the design, implementation, analysis, and writing of this study. The authors thank John Dzamba, Claude Mattox, Brian Robinson, and William Wratten for their assistance with the data analysis. Requests for reprints should be sent to Robert L. Dipboye, Rice University, Department of Psychology, Houston, Texas 77001. AMERICAN PSYCHOLOGIST • FEBRUARY 1979 • 141 Copyright 1979 by the American Psychological Association, Inc. 0003-066X/79/3402-0141$00.75 problems. Unsuspected interactions in real life may nullify or even reverse conclusions reached in the laboratory" (p. 730). Similarly, Bouchard (1976) observed that "laboratory experiments seldom deal adequately with boundary conditions or context factors and therefore, lend themselves to unjustified and often erroneous extrapolations" (p. 364). In contrast with these arguments, Fromkin and Streufert (1976) argued that "the artificiality of laboratories is being unjustly elevated to the status of a fatal flaw" (p. 433). Furthermore, they asserted that instead of reinforcing the myth that laboratory settings seldom yield data which are relevant to real world problems, it is proposed that laboratory settings merely impose identifiable limitations upon the range of criterion situations to which a particular set of laboratory findings may be practically applied, (p. 442) Despite the firm convictions held by proponents and critics, many of the arguments for and against the external validity of laboratory research are based on stereotypes rather than data. One such stereotype has been that a field setting, because it is natural to the subject and not contrived, automatically provides for more generalization of results than an artificial laboratory setting does. However, the problem of external validity is one of making inferences not only from settings but from actors and the behaviors of these actors (Runkel & McGrath, 1972). Critics of laboratory research often place inordinate weight on the setting, in discussions of external validity, to the neglect of actors and their behavior. For instance, some have assumed that because the setting in field research is typically an actual organization, data collected in such a setting must be more generalizable to other organizations than are data collected in the laboratory. However, there are differences among organizational settings that are as large as the differences that exist between an organizational setting and a laboratory. Rather than assuming that field research in the abstract is more generalizable, there is a need to determine the types of organizations with which field researchers have been concerned. The belief that laboratory research in industrial and organizational psychology is less externally valid than field research also results from the tendency to infer from the setting characteristics of the actors and their behaviors. Such inferences are often without empirical support. For example, the assumption has been made that field research involves subjects who are more representative of the working population than the college students typically used in the laboratory. But is this assumption correct? Instead of rejecting laboratory research on the basis of an untested stereotype, we need to examine more carefully who the participants are in laboratory and field research. Another assumption has been that research in field settings has as the focus of investigation natural behavior, defined by Tunnell (1977) as behavior that is not established or maintained for the sole or primary purpose of conducting research; the behavior is part of the person's existing response repertoire. Whether the procedure used to record the behavior is human or mechanical (e.g., videotapes, audiotapes, explicit behavioral checklists) the essential naturalness of the observation is maintained. Self-reports should not be considered natural behaviors unless they are made by the person in real life. (pp. 426-427) The critics of laboratory research often seem to assume that the dependent variables examined in the field are more natural and thus more externally valid than the dependent variables examined in the laboratory. But what are the dependent variables used in laboratory and field research? Again, arguments for and against the external validity of laboratory research have been based on assumptions rather than data. The issue of the relative external validity of laboratory and field settings is an important one not only to industrial and organizational psychologists but to all psychologists conducting laboratory and field research or utilizing the findings of such research. A more objective analysis of the issue of external validity is needed than has appeared in previous literature. In this article, our intent was to test empirically some of the assumptions underlying the belief that findings from field research are more generalizable to other settings, behaviors, and actors than are the findings of laboratory research. Of course, one cannot conclude that any one study or set of studies possesses complete external validity (Campbell & Stanley, 1963, p. 17), but one can discuss the relative limits that seem to exist for generalizing findings from research using different strategies. The intent of this article is to assess empirically the limits that seem to exist for recent laboratory and field research in industrial and organizational psychology. In order to achieve this objective, articles in three major journals were analyzed for content to determine the types of organizations, persons, and behaviors studied. 142 • FEBRUARY 1979 • AMERICAN PSYCHOLOGIST