AM 69-16 

METHODOLOGY IN THE ASSESSMENT OF COMPLEX 

PERFORMANCE: THE EFFECTS OF SIGNAL RATE ON 

MONITORING A STATIC PROCESS 

Approved by 

~~ J. ROBERT DILLE, M.D. 

W. Dean Chiles, Ph.D. 

Cheryl B. Bruni 

Robert A. Lewis 

Released by 

P. v. SIEGEL, M.D. 

CHIEF, CiviL AERoMEDICAL 

INSTITUTE 

FEDERAL Am SURGEON 

August 1969 

Department of Transportation 

FEDERAL AVIATION ADMINISTRATION 

Office of Aviation Medicine 
Qualified requesters may obtain Aviation Medical Reports from Defense Documentation 

Center. The general public may purchase from Clearinghouse for Federal Scientific and 

Technical Information, U.S. Dept. of Commerce, Springfield, Va. 22151. 
METHODOLOGY IN THE ASSESSMENT OF COMPLEX PERFORMANCE: 

THE EFFECTS OF SIGNAL RATE ON MONITORING A STATIC PROCESS 

I. Introduction. 

The human operator is frequently required to 

perform monitoring functions as a part of his 

job. In a significant number of aviation-related 

jobs, the signal to which the man must be alert 

and to which he must respond rapidly is in the 

nature of a discrete change in any one of the 

displays he is required to monitor. Any particular such display indicator ·assumes a given 

state and may remain in that state for a relatively long period of time, as, for example, an 

engine fire warning light in an aircraft cockpit. 

Should the indicator change its state, e.g., the 

warning light come on, the operator must respond quickly with the action appropriate to the 

emergency condition signaled by the onset of the 

light. It is rare that a man's real-world job calls 

for nothing other than monitoring sur,h displays. 

Typically, in the field of aviation, the operator 

will be engaged in a number of other tasks that 

make varying demands on his performance 

capacities. 

Much of the laboratory research that has been 

conducted on the ability of man to perform 

tasks requiring the monitoring of a static process 

has been of the nature of the classical reaction 

time experiment. In this kind of experimental 

situation, the subject is required to execute only 

one response to one specific stimulus, and, :frequently, he is given an alerting signal that tells 

him that a stimulus will appear shortly. The 

situation may be somewhat more complex; he 

may be required to choose one of two or more 

possible responses to two or more stimuli. The 

important point to be made as regards the value 

of such research to the prediction of complex 

performance in a work context is that in this 

kind of experiment, the subject can devote essentially his entire attention to the task at hand. 

Thus, the properties of monitoring behavior 

have not been investigated systematically when 

1 

that behavior is exercised in the context of a 

complex performance situation that places demands on the operator comparable to those placed 

on the pilot or air traffic contrvller. One of the 

important pieces of information needed relates 

to the effects on monitoring performance of the 

rate at which events take place within the monitoring task as a function of the number and 

nature of other tasks the man is required to perform concurrently. Of equal interest are the 

effects of monitoring task signal-rate on the performance of the concurrently presented tasks. 

An important methodological problem that relates closely to the above question concerns the 

achievement of satisfactory statistical reliability 

of measures of monitoring performance with 

relatively short periods of measurement. It is 

widely accepted that the reliability of a measure 

will increase as the number of observations 

(responses) on which that measure is based increases. However, when the rate of presentation 

of stimuli exceeds some value, the task will 

change in nature so that it can no longer be 

thought of as requiring monitoring behavior. 

When this occurs, the performance of the monitoring task will change, and, if the subject assigns a high enough priority to monitoring, 

something else that he is required to do concurrently must certainly suffer. 

Thus, the study to be reported here is concerned with three questions : 

1. What is the effect of signal rate on the 

performance of a task involving the monitoring 

of a static process when that task is performed 

concurrently with other tasks? 

2. What are the effects on concurrently performed tasks when the rate at which signals are 

introduced on the monitoring tasks is varied? 

3. What is the effect of signal rate on the 

reliability of response times in the performance 

of the monitoring task? 
II. Method. 

Subjects. The 10 subjects who served in this 

experiment were college students who had received extensive preliminary training and experience on the battery of tasks used. For purposes 

of efficiency, they were trained and tested as two 

five-man groups. They were paid for their 

services at a rate of approximately $9.00 per 

4-hour session. They were also offered a bonus 

of $2.00 for a given session if their performance 

"equalled or exceeded the level" they had previously demonstrated themselves to be capable of 

achieving. However, knowledge of having earned 

the bonus on any given session was withheld 

until the end of the experiment. 

Apparatus. The apparatus used in this experiment has been described in detail elsewhere1 and, 

therefore, will be described only briefly here. 

The subjects performed a total of five tasks presented in six different combinations as indicated 

in Table 1. 

TABLE 1.-Basic 2-Hour Task Performance Schedule 

Task 

00 15 30 

Warning-lights monitoring ________ XXX XXX XXX 

Probability monitoring ___________ XXX XXX XXX 

Arithmetic computation __________ XXX XXX 

Code-lock solving ________________ XXX 

Target identification _____________ 

Level of demand on performance low med 

The primary task of interest in the present 

study was called "warning lights." One aspect 

of this task was represented by five red lightsone at each corner and one in the middle of the 

panel. A button was located below each light, 

and the subject was instructed to push the button 

associated with a given light should that light be 

illuminated; pushing the button extinguished the 

light. The second aspect of the task consisted 

of five green lights paired with the red lights. 

In this case, the subject was instructed to push 

the button below a given green light should that 

light go out. For each aspect of the warning 

lights task, response times were measured in 

tenths of a second from the onset (or offset) of a 

light until the subject returned the light to its 

normal condition by pushing the appropriate 

button. 

The second task, called "probability monitoring," required the subject to scan four randomly 

fluctuating meters to determine if the average of 

the pointer positions of any meter had deviated 

from zero (12 o'clock position). The signal to 

which the subject was to respond was a shift of 

the distribution of pointer positions of a given 

meter from a mean of zero to a mean that was 

25 units to the right or the le:ft of zero. If the 

subject suspected a bias to be present on a parhi 

2 

Minutes 

45 60 75 90 105 120 

XXX XXX XXX XXX XXX 

XXX XXX XXX XXX XXX 

XXX XXX XXX 

XXX XXX 

med med hi med low 

ticular meter, he tested the hypothesis by throwing a three-position, spring centered, lever-type 

switch in the direction of the suspected bias. 

The pointer would automatically come to rest on 

its correct mean, thus, giving him immediate 

feedback as to the accuracy of his hypothesis. 

Time was recorded to the nearest second from 

the introduction of a bias until the appropriate 

response removed the bias or until that bias 

was replaced by a new bias on the same or some 

other meter. Signals were introduced on this 

task at an average rate of 15.5 signals per hour, 

and only one meter was biased at any given time. 

Two measures were computed: Mean detection 

times for those signals that were detected and 

percent of signals detected. 

The third task was called "mental arithmetic." 

It required the subject to sum two three-digit 

numbers and subtract a third three-digit number 

from the obtained sum. The subject entered his 

answer by appropriate manipulation of a set of 

three decade pushbuttons for the first three digits 

and a three-position, center-off switch to indicate 

whether the fourth digit was a "O" or a "1." 

Problems were presented at a rate of three per 

minute; performance was scored in terms of the 

percentage of problems answered correctly. 
The fourth task, called "target identification,': 

required the subject to view a standard "target 

image" and then decide whether the first, second, 

or neither of two comparison images was the 

same as the standard image. The task was made 

somewhat more difficult by random distortion o£ 

the comparison images. The subject indicated 

his answer by depression of the appropriate one 

of three buttons marked "1," "2," and "N" 

(Neither). Problems were presented at a fixed 

rate of two per minute. Performance was measured in terms of the percentage of problems 

answered correctly. This task also had a "crew" 

component. After one of the subjects, who was 

designated as the crew commander, had made his 

response as an individual, an auxiliary display 

was illuminated at his crew position showing the 

choices of each member of the crew (including 

his own). Making whatever use he wished of this 

information about the crew responses, the commander made a selection of an answer "on behalf 

of the crew." The commander's choice was then 

displayed at each individual crew position. Only 

the crew commander received direct feedback as 

to which answer was correct, and it was his option as to whether or not to pass this information 

on to the remainder of the crew. Both crew 

commanders almost always indicated the correct 

answer to the crew. Performance on the task, 

both individual and crew, was measured in terms 

of the percent of correct responses. 

The fifth task, called "code-lock," involved 

group performance in the solution of problems. 

The subjects were required to find the correct 

sequence in which each subject should push the 

code-lock button located on each subject's panel 

to illuminate a green light indicating that the 

problem had been solved. The onset of a red 

light on each subject's panel indicated that a 

problem was present. When the subject who was 

number one in a particular sequence pushed his 

button, the red light would go out and remain 

out as long as the subsequent responses made were 

in the correct sequence. A button pushed out of 

sequence would re-illuminate the red light and 

the problem would automatically reset to the 

beginning, thus requiring the subjects already 

identified to push their buttons again before the 

search could continue. The subjects were trained 

to follow a standard search procedure so that 

redundant responses could be more readily prevented. A 30-second delay followed the solution 

3 

of a given new problem after which the same 

problem was presented again and the subjects 

were to enter the solution as rapidly as possible 

without error. After another 30-second delay, a 

new problem was presented. The subjects used 

an earphone intercom system to coordinate performance of the task, and the crew commander 

was responsible for maintaining an efficient attack on the problems. Although measures were 

made of the time required to solve the problems, 

the number of errors, and the total number of 

responses made by all subjects, the code-lock data 

were not subjected to statistical analyses; since 

the unit of analysis was the crew, the one degree 

of freedom available for the appropriate error 

term made any such analysis of questionable 

value. Thus, the major functions served by this 

task were to provide a workload factor and to 

introduce an element of subject interaction. 

Procedure. Prior to the beginning of the experiment, subjects were given extended training 

on the task complex and served in an experiment2 

involving three 4-hour sessions in which different 

signal rates were used on the probability monitoring task. 

The e3'perimental phase of the study consisted 

of three, 4-hour sessions for each of the two 

groups of subjects. The first group (Able crew) 

was tested on the morning of the 11th, 15th, and 

22nd of March. The second group (Baker crew) 

was tested on the afternoons of the 13th, 19th, 

and 21st of March. (The month is of significance only in that the specific schedule of test 

sessions resulted from inclement weather.) On 

the first test session, each group was presented 

warning light signals at the same rate used 

during training and during the earlier experiment. The average signal rate was 17 signals 

per hour; the mean intersignal interval was 212 

seconds and the range was 119 seconds. On the 

second session, the average signal rate was 10.3 

signals per hour; the mean intersignal interval 

was 348 seconds; the range was 195 seconds. On 

the third session the average signal rate was 22.5 

signals per hour with a mean intersignal interval 

of 160 seconds and a range of 90 seconds. 

Performance was measured on the combinations 

of tasks shown in Table 1, each session consisting 

of two of these 2-hour programs in succession. 

Although no formal rest period was provided 

in the 4-hour session, subjects were granted per-
mission, on reque8t, to leave their duty stations 

one at a time £or the purpose o£ going to the rest 

room or getting a drink o£ water. 

III. Results and Discussion. 

Warning Lights Monitoring. A separate nonparametric statistical analysis (Friedman's twoway analysis o£ variance by ranks) \vas applied 

to the data £or response times to the red lights 

£or each o£ the six combinations o£ tasks under 

which performance was measured. The mean 

response times £or each signal rate and each task 

combination· are shown in Table 2 ; the means 

£or the data combined over the first 2 hours o£ a 

session, the second 2 hours, and the entire 4 hours 

are also shown. In no case did the resultant 

statistic (Chi -square r) even approach significance. 

TABLE 2. Mean Response Time in Seconds to Red 

Lights 

Signals/hour 

Task Combination X2r p 

17.0 10.3 22.5 

------

Monitoring only ____ . 9 .9 1.0 ------ -----

Arithmetic __ - ______ 1.0 1.3 . 9 ------ ----- Arith & code-lock __ 1.2 1.0 1.0 ------ ----- Code-lock __________ 1.0 1.0 1.0 ------ ----- Target ident & 

code-lock ________ 1.0 1.2 1.4 3.65 . 2 0 

Target ident_ ______ 1.0 1.0 1.0 ------ ----- 1st 2 Hours ________ 1.0 .9 1.0 ------ -----

2nd 2 Hours _______ 1.0 1.2 1.1 ------ ----- 4 Hours ___________ 1.0 1.0 1.0 ------ -----

The attention value o£ the illumination o£ a 

red light was reported by all subjects to be relatively high, and it was unusual £or a signal to 

go undetected £or even as long as 5 seconds. 

Presumably, the scanning habits developed by 

the subjects during the course o£ training at a 

rate o£ 17 signals per hour were adequate to 

prevent performance changes at the higher as 

well as the lower rates o£ signal presentation. 

The data £or mean response times to the offset 

o£ green lights are shown in Table 3. Again, 

none o£ the statistical tests approached significance. The attention value o£ the offset o£ a 

green light is not as high as that o£ the onset o£ 

a red light. This is seen clearly in the £act that 

4 

the grand mean response time £or red lights was 

1 second whereas the grand mean £or green lights 

was 3.8 seconds. However, as with the red lights, 

the most likely explanation o£ the lack o£ an 

effect o£ signal rate is that the levels o£ training 

o£ the subjects had resulted in the development 

o£ efficient scanning habits with respect to the 

detection o£ extinguished green lights. 

TABLE 3. Mean Response Time in Seconds to Green 

Lights 

Signals/hour 

Task Combination X2r p 

17.0 10.3 22.5 

------

Monitoring only ____ 4.1 2.0 1.7 ------ ----- Arithmetic _________ 4.2 5.0 3.7 ------ ----- Arith & code-lock __ 5.5 4.0 3.7 ------ ----- Code-lock_------- __ 3.0 2.6 3.5 ------ ----- Target ident & 

code-lock ________ 4.3 3.7 7.9 ------ ----- Target ident _______ 3.1 6.7 4.4 ------ ----- 1st 2 Hours ________ 2.9 4.1 3.4 ------ ----- 2nd 2 Hours ________ 5.0 4.1 3.8 ------ ----- 4 Hours ___________ 3.7 4.1 3.6 ------ -----

Thus, changes in signal rate within the range 

o£ values used in the present experiment do not 

affect the speed with which subjects detect signals. This finding holds £or both responses to 

the onset o£ a red light and the offset o£ a green 

light. 

Probability 111 onitoring. Data £or the percentage o£ detections o£ meter biases are not presented since essentially all signals were detected, 

and those £ew signals that were missed were scattered across the three rates at which warning 

lights signals were presented. The finding that 

nearly all biases were detected is consistent with 

previous research with this task with test sessions 

o£ 4 hours or less.1 2 

The mean detection times £or the three experimental conditions are shown in Table 4. 0£ the 

nine statistical tests applied to these data, significant differences were found in £our cases. The 

first significant statistic was £or the "monitoring 

only" condition; the slow rate o£ presentation o£ 

warning lights signals resulted in the longest detection times £or meter biases, and the £ast rate 

o£ presentation was the best. However, the di£-
ference between the two extreme values was only 

3 seconds or approximately 12% of the mean detection time at the fast rate of signal presentation. It is of some interest to note that a test of 

the two extreme values resulted in a Wilcoxon 

T that was not significant (N=10; T.o5 =8; obtained T=9). 

TABLE 4. Probability Monitoring Mean Detection 

Times in Seconds 

Signals/hour on 

warning lights task 

Task X2r p 

combination 

(Training) 

17.0 10.3 22.5 

--------

Monitoring only _________ 25 27 24 6.2 <.05 

Arithmetic _____ 35 34 30 ------ ------

Arith. & codelock ________ - 41 41 36 ------ ------

Code-lock ______ 41 31 31 7.8 <.05 

Target ident. & 

code-lock ____ 32 45 37 ------ ------ Target ident ___ 39 39 34 ------ ------ 1st 2 Hours ____ 33 36 29 ------ ------ 2nd 2 Hours ____ 36 34 33 6.2 <.05 

4 Hours _______ 35 35 31 9.8 <.01 

The differences in bias detection times across 

warning lights signal rates were also significant 

for the condition in which the probability monitoring task was performed concurrently with 

warning lights and code-lock. Here, the differences between the mean detection times at the 

medium signal rates on warning lights and both 

the fast and slow rates were substantial (10 

seconds in each case). Individual Wilcoxon T's 

showed the middle and slow rates to differ significantly (P<.05) but not the middle and fast 

signal rates (P>.05). 

The differences across signal rates were also 

significant in the case of the combined data for 

the second 2 hours of the sessions and for the 

entire 4 hours of testing. However, it should be 

noted that these latter t-wo tests are not independent of the other tests (or of each other) in that the 

data used in the tests of the individual task 

combinations are contained in the 2-hour and 

· 4-hour data. In the case of the data for the 

second 2 hours, the rank orders of the mean detection times at the medium and slow rates are 

5 

reversed as compared to the ordering during the 

first 2 hours; performance was better under the 

medium signal rate during the first 2 hours 

(though not significantly) but was better under 

the slow signal rate during the second 2 hours. 

The detection times were the same for these signal rates for the 4-hour data. From these fact~ 

it can be inferred that increasing the rate at 

which warning lights signals are introduced 

from 10.3 to 17.0 signals per hour probably does 

not have a real effect on the detection times for 

probability biases. Increasing the rate from 17 

signals to 22.5 signals per hour probably does 

have an effect. Clearly, more data are required 

to determine whether or not the effect is real and, 

if it is, to determine the warning lights signal 

rate at which the effect is produced. 

Inspection of Table 4 suggests that the obtained significance for the performance condition 

in which code-lock and the monitoring tasks 

were active concurrently may have been the result of unusuallly poor performance for the session involving the medium rate of signal presentation on the warning lights task. Fortunately, 

another set of data were collected in an earlier 

study under the same experimental conditions 

that obtained in the present study. These data 

were from an experiment (in which these subjects served) on the performance effects of the 

rate at which signals were presented on the 

probability monitoring task. 2 The signal rates on 

all tasks \vere identical in the two experiments 

for the "training" (medium rate) conditions. 

Therefore, a second set of statistical tests was 

carried out in which these data from the earlier 

experiment were substituted for the data collected at the 17 -signals/hour rate in the present 

experiment. The tests for the monitoring only, 

the second 2 hours, and the entire 4 hours still 

showed signal rate on the warning lights task to 

have a significant effect on detection times on the 

probability monitoring task. However, in the 

case of the code-lock task, the effect was not significant. 

Responses to warning lights were elicited a 

little over twice as often at the fast signal rate 

than at the slow signal rate. The result of this 

could very well be that the general level of alertness of the subject was higher at the fast signal 

rate than at the slow rate. This effect would 

be most pronounced during the parts of the session in which only the monitoring tasks were 
being performed. There is another possible effect 

of the increased rate of responding at the fast 

rate of presentation of warning lights signals. 

Although substantive data are not available, observation of and discussion with subjects in previous experiments have led to the tentative 

conclusion that responding to a warning light 

serves as a stimulus to scan the four probability 

meters. Thus, since subjects respond to virtually 

every signal, then the more frequently signals are 

introduced on the warning lights task, the more 

frequently will the subjects scan the probability 

meters. This scanning would be for the most part 

in addition to the scanning the subjects do as a 

result of habits developed through practice. 

Clearly, these two possible modes of operation 

of signal rate on the warning lights task are not 

mutually exclusive and probably both were operating. 

Mental Arithmetic and Target Identification. 

The "percentage correct" data for both of these 

tasks are summarized in Table 5 for each signal 

rate condition and each task combination. None 

of the statistical tests revealed the rather small 

differences across warning lights signal rates to 

be significant. 

TABLE 5. Mean Percent Correct on Arithmetic and 

Target Identification Tasks 

Signals/hour 

on warning lights 

Task combination X2r p 

17.0 10.3 22.5 

--------

Arith. only 

1st 2 Hours ______ 91.1 92.2 92.7 ------ ----- 2nd 2 Hours _____ 91.8 90.7 93.0 ------ ----- 4 Hours _________ 91.5 91.5 92.9 ------ ----- Arith. & code-lock 

1st 2 Hours ______ 87.0 84.9 87.8 ------ ----- 2nd 2 Hours ______ 87.4 86.4 91.0 ------ ----- 4 Hours _________ 87.2 85.7 89.4 ------ ------ T arget ident. only 

1st 2 Hours ______ 86.3 88.0 86.3 ------ ------ 2nd 2 Hours ______ 90.3 86.3 86.0 ------ ------

4 Hours _________ 88.3 87.2 86.2 ------ ------ Target ident. & 

c ode-lock 

1st 2 Hours ______ 72.7 75.3 81.0 ------ ------ 2nd 2 Hours ______ 79.7 80.3 78.6 ------ ------ 4 Hours _________ 76.2 77.8 79.8 ------ ------

6 

All of the previous work with these tasks supports the appropriateness of the assumption that 

subjects tend to assign relatively high priorities 

to the performance of the arithmetic and the 

target identification tasks. Perhaps the major 

reason is that the subject gets immediate feedback as to how well he is doing on arithmetic 

and usually gets feedback (from the commander) 

that permits him to determine how well he is 

doing on the target identification task. Another, 

social incentive for doing well on the target 

identification task derives from the fact that the 

commander sees what response a given subject 

has made on each problem, and the commander of 

the Baker crew in particular urged a subject to 

do better if that subject was wrong on very many 

responses. Thus, there is no reason to expect the 

signal rate of the warning lights task to have a 

facilitative effect on the performance of either 

arithmetic or target identification. On the other 

hand, the workload-producing characteristics of 

the warning lights task are not such that there 

would be any reason to expect the high signal 

rate condition to interfere with either task. 

Stability of Warning Lights Data. The relative stabilities of the response times to the red 

and green light signals across the three rates of 

signal presentation were evaluated by comparing 

the reliability coefficient for each rate. Specifically, the correlation between the mean response times for the first 2 hours of performance 

and the second 2 hours of peformance for each 

signal rate was computed. Both Spearman rank 

correlation coefficients and product-moment coefficients were computed. 

The Spearman rank coefficients are shown in 

Table 6. For both the red and the green lights, 

the coefficients at the two faster rates of presentation were significant at better than the .05 level 

of confidence using one-tailed tests. One coefficient, green lights at the rate of 17 signals/hour, 

was significant at the .01 level. As regards this 

latter coefficient, the obtained value of .94 is the 

highest reliability coefficient ever obtained for 

this specific task; typical values previously obtained with larger subject samples have ranged 

from .50 to .70 for data based on a greater total 

number of responses per subject. 

The product-moment coefficients for the two 

faster rates were also significant at better than 

the .05 level of confidence (one-tailed tests), and 

one coefficient, red lights at the 17 signals/hour 
rate, was significant at the .. 025 level of confidence. For both the product-moment and the 

Spearman rank coefficients, the reliabilities at 

the slow signal rate (10.3 signals/hour) were low 

and not significant. 

TABLE 6. Half-Session Reliability Coefficients: 

Spearman's Rank Correlations 

Lights 

Red ________________ 

Green ______________ 

One-tailed test: 

*.05 level of confidence 

**.01 level of confidence 

17.0 

. 69* 

. 94** 

Signals/hour 

I 

10.3 

.18 

-.07 

22.5 

. 62* 

. 59* 

TABLE 7. Half-Session Reliability Coefficients: 

Product-Moment Correlations 

Lights 

17.0 

Red ________________ .772** 

Green ______________ • 626* 

One-tailed test: 

*.05 level of confidence 

**.025 level of confidence 

Signals/hour 

10.3 

. . 200 

.027 

22.5 

. 689* 

. 705* 

The Spearman-Brown prophesy formula was 

used to predict the reliabilities to be expected for 

the full 4 hours of testing. In the case of the 

red lights, the predicted reliabilities were .871, 

.280, and .815 for the medium, slow ,and fast rates 

of signal presentation, respectively. The prepicted reliabilities for the two faster rates are 

significant at the .025 level of confidence. For 

the green lights, the predicted 4-hour reliabilities 

were .69, .053, and .826. Again, the reliabilities 

at the two faster rates are significant at the .025 

level o£ confidence. 

Most of the previous research with this battery 

of tasks used the slow rate of presentation of signals on the warning lights task.1 However, in 

that research, subjects were typically tested for 

7 

a total of 12 hours (or more) per day, and the 

unit of analysis for the more important variables 

was the mean of a full day's performance. Thus, 

the number of responses on which the data entering into the calculations were based was large 

enough that the apparently low reliability of a 

2-hour session was not a problem. 

The present data clearly suggest that an important gain in the stability of the response data 

for the warning lights task can be realized by 

using a signal rate on the order of 17 signals/ 

hour. Although in theory a rate of 22.5 signals/hour should 'yield even more stable data, 

this was not demonstrated by the present study. 

IV. Summary and Conclusions. 

This study was concerned with the effects of 

the rate of presentation of stimuli on a task involving the monitoring of a static process when 

that task is performed concurrently with a variety of tasks designed to assess aviation-related 

performance. The task (called warning lights 

monitoring) consisted of two aspects; the subject responded to the onset of normally-off red 

lights and to the offset of normally-on green 

lights by depressing a button to return the light 

to its normal state. Five red lights and five 

green lights were located in pairs on the subject's performance panel-one red light and one 

green light at each corner and one pair in the 

middle. Three signal rates were used: 10.3 signals/hour, 17.0 signals/hour, and 22.5 signals/ 

hour. Each signal rate was presented during a 

different 4-hour session of testing. The subjects 

performed on a standard schedule involving four 

other tasks presented in different combinations 

to vary both the level and the nature of the 

workload imposed. The other tasks involved 

monitoring (of a dynamic process), mental arithmetic, visual discrimination, and group problem 

solving. Subjects were given extensive preliminary training on the task complex prior to the 

beginning of testing. 

The only task for which performance varied 

significantly across the three rates of signal presentation was the task requiring the monitoring 

of a dynamic process ( probability monitoring). 

This task required the subject to determine 

whether or not the mean position of the continuously fluctuating pointer of any of four 

meters had deviated from zero. Significant dif-
ferences were found when the probability monitoring task was performed with only the warning lights task, and when performed with the 

warning lights task plus the group problem 

solving task. The effect of signal rate was also 

significant for the data combined across all task 

combinations for the second 2 hours of the test 

session and for the combined data for the entire 4-hour session. The explanation offered for 

the effect of the signal rate of the warning lights 

task on the performance of the probability monitoring task is; ( 1) the increased rate of response to warning lights at the higher signal 

rates served to increase the level of alertness of 

the subjects particularly during the periods when 

only the two monitoring tasks were performed, 

and (2) based on previous observations, the warnlights signals tended to serve as stimuli for the 

subjects to scan the meters on the probability 

monitoring task; thus, the more frequently signals were introduced, the more frequently the 

subjects scanned the four probability meters. 

It was also found that the reliability coefficients for the warning lights task (first 2 hours 

of a session versus the second 2 hours) were 

substantial and significant for the two faster 

signal rates; the coefficient was low and nonsignificant for the slower rate. 

The following conclusions are drawn: 

1. The rate at which signals are presented 

on a task involving the monitoring of a static 

process does not significantly affect the speed 

8 

with which signals are detected. The major 

assumptions underlying this conclusion (as well 

as the remaining conclusions) are: (a) the attention value of the signals is relatively high, (b) 

the signal rate is within the aproximate range 

of from 10 to 23 signals per hour, and (c) the 

task is performed as a part of a task complex. 

2. The concurrent perfomance of other tasks 

on which subjects would be expected to place 

high priorities is not affected by variations in 

signal rate on a static monitoring task. Clearly, 

above some signal rate (presumably much higher 

than that used in the present study), direct interference would result. Either the concurrently 

performed task would suffer or the monitoring 

task would be largely ignored. 

3. Concurrent performance of a task involving 

the monitoring of a dynamic process may be facilitated by increasing the signal rate of the 

static monitoring task. This assumes that the 

signals on the dynamic monitoring task are of 

relatively low attention value and require close 

attention to determine their presence. 

4. Increased rate of presentation of signals on 

a static monitoring task will result in increased 

reliability of the response measures for test 

periods on the order of 2 to 4 hours. Thus, more 

reliable assessment of this aspect of complex performance, as it is related to the demands placed 

on the human operator in aviation operations, 

is now possible. 
REFERENCES 

1. CHILES, W. D., ALLUISI, E. A., and ADAMS, 0. S. : 

Work schedules and performance during confinement. 

Human Factor-s, 10:143-196, 1968. 

2. CHILES, W. D., BRUNI, CHERYL B., and LEWIS, R. A. : 

Methodology in the assessment of complex human 

9 

performance: the effects of signal rate on monitoring 

a dynamic process. FAA Office of Aviation Medicine, 

Report No. 69-6, 1969.