Toward a Valid View of Human Factors Research: Response to Vicente (1997)

INTRODUCTION Vicente (1997) presents several interesting ideas regarding human factors research and the relationship between basic and applied research. Many of the issues regarding basic and applied work have been discussed recently in textbooks (e.g., Payne & Conrad, 1997), chapters (e.g., Payne, Conrad, & Hager, 1997), and articles (e.g., Koriat & Goldsmith, 1994), and we feel that both these sources and Vicente's commentary provide important reminders of the importance of both types of research. Although we agree with some of Vicente's views, we differ on at least two important points. First, we question the utility and accuracy of Vicente's four-type categorization scheme for human factors research. Second, we disagree with his characterization of the work of Payne, Lang, and Blackwell (1995) and the alleged contradiction between their work and that of Hansen (1995). The first point concerns Vicente's prescriptions regarding human factors research. Vicente characterizes human factors research as including four "types" of research. We find this characterization problematic in two ways. First, the criteria used to classify research into these four types are vague, insofar as no operational definitions are given. The definitions Vicente provides are all highly relative; for instance, a Type 1 experiment is a "highly controlled laboratory experiment," and a Type 2 experiment is a "less controlled but more complex" experiment (p. 324). Such definitions make it highly difficult to categorize any single study objectively. For example, Vicente cites the Gould et al. (1987) study of reading from paper versus CRT as Type 1 research. However, if one were to compare the work of Gould et al. with a tightly controlled experiment examining eye movements in reading, the work of Gould et al. might be considered Type 2 research. Second, we take issue with the picture Vicente paints concerning the relative strengths and weaknesses of each type of research. For example, he argues that Type 2 research is more likely to generalize than Type 1 research because the former is more representative of operational settings. Vicente states that this assertion is "a statistical fact" (p. 326). We agree that the more closely an experimental setting emulates a specific real-world setting, the more likely the results are to apply to that specific setting. However, the extent to which research findings can or should be generalized across settings depends on the extent to which critical factors controlling behavior are common across the original setting and the setting to which one generalizes. If a basic laboratory study identifies factors that influence performance, then these factors will allow one to make predictions about the real world. Vicente misses the point that knowing the extent to which a "Type 2" experiment is representative requires knowing which factors determine representativeness and the setting for which one wishes to generalize. It is not a "statistical fact." For applied researchers it is absolutely essential that the results of studies generalize beyond the original research setting. If studies lack generalizability, then with each new operational setting one is forced to conduct research specific to that setting. Such an approach is expensive and inefficient. Our second major point concerns the characterization of the research by Hansen (1995) and Payne et al. (1995) that Vicente uses to motivate his arguments. Vicente quotes statements from these two articles that, when taken at face value and out of context, appear to be at odds. Vicente asserts that "there is a glaring contradiction in the assertions made in these two papers" and that "in fact" one of the two papers "is incorrect" (p. 324). As authors of one of the papers in question, we think it is important to set the record straight. In our opinion, the "glaring contradiction" between Hansen and Payne et al. simply does not exist. …