Inspectors' Interpretation of Instructions in the Aircraft Maintenance Domain

Previous research has identified the importance of instructions in the complex domain of airline maintenance. Written instructions, or workcards, a& the primary sonrces of information in this environment. The workcards combine information from a particular airline’s maintenance manual (the general policies and procedures of that airline) with other sources, including the aircraft manufacturer’s repair manuals and directives issued by regulatory authorities. It is generally accepted that all inspectors have the same interpretations of what is expected ,of them, and that differences in performance reflect real differences in inspector abilities. However, results from a previous study suggest that inspectors had different interpretations of what specifically was to be performed during the task, which guided both the strategy used to conduct the inspections and the results of these inspections (Wenner, Wenner, Dory and Spencer, 1997). This study examined how choice of wording and level of detail impacted workcard interpretation. Twenty-eight airline inspectors, from a major airline (mean age = 50.6 and mean years of experience = 30.5) and repair station (mean age = 42.3 and mean years of experience = 16.7), participated in this study. Each inspector was asked to complete. a four part written questionnaire at their facility during normal working hours. The first three parts of the questionnaire were the same for all inspectors. Part 1 asked the inspectors to rank sets of five instructions in terms of how much time they would need to perform each inspection, or how indepth the inspections should be. Part 2 asked the inspectors to determine if inspection instructions were the same or different, and to explain why they had this opinion. Part 3 gave the inspectors four sets of instructions, varying in the amount of detail included. Inspectors were then asked a series of questions (e.g., Which workcard (s) require you to look for delamination?), and were asked to select the workcards which best answered each question. For the fourth part, each inspector was given one of four different workcards (seven inspectors used each workcard) for the same task. Again, the workcards varied in the amount of detail included. All inspectors were then asked to complete the same set of questions based on this workcard. The results from this study indicated there are differences in workcard interpretation, even among highly experienced inspectors. In Part 1, the coefficient of concordance for the rankings of the various inspections was only .62, suggesting that there was not uniform agreement between the inspectors. In Part 2, there was a consensus among the inspectors that both a detailed visual check and an intensified visual check described a more in-depth inspection than a general visual check. However, there was only partial agreement on the explanations of other terms. Again, this suggests that there is not a uniform understanding of the different levels of inspection. Furthermore, the results from Part 3 suggest that the inclusion of different amounts and types of information does affect the way inspectors see the task. Even though the four workcards addressed exactly the same task, only three inspectors indicated that the tasks were the same on all eight questions asked. The responses from the other inspectors indicated that their interpretation of the task was impacted by the information included in the instructions. However, the results fromPart 4 suggest that inspectors are able to supplement deficiencies in the instructions with experience and domain knowledge. These resu!ts suppoti the finding of a laboratory study that was conducted to further explore the effects of instructions on search performance and strategy. The results from that study are summa ized in Wenner (2000). The overall results from these two studies suggest that although instructions can influence performance on a search task, experience and expectations also play a role in how the instructions are used to carry out the task.