A Process-Oriented Nuts and Bolts Tutorial for 

Implementing Manpower and Personnel 

Integration (MANPRINT) Evaluations 

by Otto H. Heuckeroth 

ARL-TR-4637 November 2008 

Approved for public release; distribution is unlimited. 
NOTICES 

Disclaimers 

The findings in this report are not to be construed as an official Department of the Army position unless 

so designated by other authorized documents. 

Citation of manufacturer’s or trade names does not constitute an official endorsement or approval of the 

use thereof. 

Destroy this report when it is no longer needed. Do not return it to the originator.
Army Research Laboratory 

Aberdeen Proving Ground, MD 21005-5069 

ARL-TR-4637 November 2008 

A Process-Oriented Nuts and Bolts Tutorial for 

Implementing Manpower and Personnel 

Integration (MANPRINT) Evaluations 

Otto H. Heuckeroth 

Human Research and Engineering Directorate, ARL 

Approved for public release; distribution is unlimited. 
ii

REPORT DOCUMENTATION PAGE Form Approved 

OMB No. 0704-0188

Public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering 

and maintaining the data needed, and completing and reviewing the collection information. Send comments regarding this burden estimate or any other aspect of this collection of information, 

including suggestions for reducing the burden, to Department of Defense, Washington Headquarters Services, Directorate for Information Operations and Reports (0704-0188), 1215 Jefferson 

Davis Highway, Suite 1204, Arlington, VA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to any penalty for failing to 

comply with a collection of information if it does not display a currently valid OMB control number. 

PLEASE DO NOT RETURN YOUR FORM TO THE ABOVE ADDRESS. 

1. REPORT DATE (DD-MM-YYYY)

November 2008 

2. REPORT TYPE 

Final 

3. DATES COVERED (From - To) 

1986–Present 

4. TITLE AND SUBTITLE 

A Process-Oriented Nuts and Bolts Tutorial for Implementing Manpower and 

Personnel Integration (MANPRINT) Evaluations 

5a. CONTRACT NUMBER 

5b. GRANT NUMBER 

5c. PROGRAM ELEMENT NUMBER 

6. AUTHOR(S) 

Otto H. Heuckeroth 

5d. PROJECT NUMBER 

62716AH70 

5e. TASK NUMBER

5f. WORK UNIT NUMBER

7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 

U.S. Army Research Laboratory 

ATTN: AMSRD-ARL-HR-MV 

Aberdeen Proving Ground, MD 21005-5069 

8. PERFORMING ORGANIZATION 

 REPORT NUMBER 

ARL-TR-4637 

9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSOR/MONITOR’S ACRONYM(S)

11. SPONSOR/MONITOR'S REPORT 

 NUMBER(S)

12. DISTRIBUTION/AVAILABILITY STATEMENT 

Approved for public release; distribution is unlimited. 

13. SUPPLEMENTARY NOTES 

14. ABSTRACT 

Manpower and Personnel Integration (MANPRINT) type concerns for military and non-military systems under development 

have been of interest as long as man has been developing systems. 

In the mid 1980s, as the MANPRINT program was being formally developed, personnel who would implement this initiative 

received a 3-week training course on the more management-oriented aspects for MANPRINT. Following this training, those 

selected to conduct MANPRINT evaluations during operational testing had to work out the data-collection procedures. Our 

role to provide MANPRINT evaluations to each new emerging system made it necessary to continually adapt implementing 

methodology to conduct those evaluations. 

Initial impetus for this report followed a request by the Intelligence and Electronic Warfare Directorate (IEWTD) at Fort 

Huachuca to develop a tutorial on how MANPRINT evaluations could be implemented. While most MANPRINT Assessments 

do well in presenting problems identified (are product-oriented), they frequently are incomplete on the detailed data collection 

methodology—the specific processes used. Based upon the MANPRINT support procedures used (or developed) by the author 

since 1986 for several emerging systems, a group-oriented training tutorial on those processes was developed. Thanks to a 

formative review of this document by the Fort Hood Field Element Chief, the tutorial presented here has been redesigned for 

use by individuals. 

15. SUBJECT TERMS

MANPRINT, manpower, personnel, training, human factors engineering, health hazards, system safety, soldier survivability, 

implementing assessment 

16. SECURITY CLASSIFICATION OF: 

17. LIMITATION 

OF ABSTRACT 

UL 

18. NUMBER 

OF PAGES 

256 

19a. NAME OF RESPONSIBLE PERSON

Otto H. Heuckeroth 

a. REPORT 

UNCLASSIFIED 

b. ABSTRACT 

UNCLASSIFIED 

c. THIS PAGE 

UNCLASSIFIED

19b. TELEPHONE NUMBER (Include area code) 

254-288-9377 

Standard Form 298 (Rev. 8/98) 

Prescribed by ANSI Std. Z39.18 
iii

Contents 

List of Figures v

List of Tables v

Acknowledgments vi

1. Introduction 1

1.1 Development of MANPRINT Implementation ...............................................................1

1.2 Purpose ............................................................................................................................2

2. Method 4

2.1 Characterization of MANPRINT ....................................................................................5

2.1.1 Manpower Domain ................................................................................................5

2.1.2 Personnel Domain .................................................................................................6

2.1.3 Training Domain ...................................................................................................7

2.1.4 Human Factors Engineering Domain ....................................................................8

2.1.5 Health Hazards Domain ........................................................................................9

2.1.6 System Safety Domain ..........................................................................................9

2.1.7 Soldier Survivability Domain ................................................................................9

2.2 MANPRINT Models for Evaluation .............................................................................10

2.3 Resources and Data Collection Procedures for Conducting MANPRINT 

Evaluations With Four Different Models ...............................................................................12

2.3.1 Use of Model 1 for MANPRINT Evaluation of the All-Source Analysis 

System (ASAS) .............................................................................................................12

2.3.2 Use of Model 2 for MANPRINT Evaluation of the Heavy Equipment 

Transporter System (HETS) ..........................................................................................83

2.3.3 Use of Model 2 for MANPRINT Evaluation of the Enhanced Position 

Location Reporting System (EPLRS) ...........................................................................96

2.3.4 Use of Model 3 for MANPRINT Evaluation of ASAS in the War Fighting 

Rapid Acquisition Program (WRAP) ..........................................................................101

2.3.5 Use of Model 4 for MANPRINT Evaluation of the Secure Mobile Anti-Jam 

Reliable Tactical Terminal (SMART-T) in the FOTE ................................................103

2.3.6 Use of Model 4 for MANPRINT Evaluation of the M1A2 in the IOTE ..........103 
iv

3. Conclusions 106

4. References 107

Appendix A. Memorandum for Defense Manpower Data Center (DMDC) to Support 

Sample Representativeness Comparisons 109

Appendix B. Sample Representativeness of Soldiers Participating in the March 1999 

DT/OT RWS LUTE 113

Appendix C. Human Factors Engineering Questionnaire for an Electronic Equipment 

Test Facility (EETF) 121

Appendix D. Human Factors Engineering Evaluation of Regency Net Equipment 161

Appendix E. Resource Material for Health Hazards, System Safety, and Risk 

Assessment Matrix 177

Appendix F. Maneuver Control System (MCS) Training Questionnaire 181

Appendix G. Maneuver Control System Workload Questionnaire 189

Appendix H. MANPRINT Survey/Interview Questions for All-Source Analysis 

System (ASAS) Block II Initial Operational Test and Evaluation (IOTE) 203

Appendix I. Noise and Temperature Measurements in and Around the HETS M1070 

Tractor 233

Appendix J. MANPRINT Biographical Questionnaire: Enhanced Position Location 

Reporting System (EPLRS) 237

Distribution List 245 
v

List of Figures 

Figure 1. Technical manual procedure for performing task being interviewed. ...........................90

List of Tables 

Table 1. First four levels of data. ....................................................................................................3

Table 2. Models for MANPRINT evaluation (but “every test is different”). ...............................10

Table 3. Footnotes referenced in table 2. ......................................................................................11

Table 4. Event design plan (EDP) for the all-source analysis system (ASAS) block 2 

Remote Work Station (RWS) limited user test and evaluation (LUTE) data collection 

plan. ..........................................................................................................................................13

Table 5. Training evaluation debrief data collection. ...................................................................17

Table 6. MANPRINT debrief for collective tasks. .......................................................................38

Table 7. Documentation of inter-operability difficulties—problems receiving/sending 

different message types among (intra/inter) battlefield functional areas (BFA). ....................68

Table 8. MANPRINT interview questions used during WRAP for ASAS. ...............................102
vi

Acknowledgments 

Thanks are extended to Dr. Middlebrooks for his review of this report and painstakingly pointing 

out that the MANPRINT support provided over the years by the author was better described as 

MANPRINT process evaluation than assessment. 

Thanks are due to contract personnel affiliated with the Operational Test Command for their 

guidance in preparation of this report. Special thanks are extended to Mr. Walter Tindell, 

Customer Support Specialist, for his repeated guidance in manipulating the syntax of Microsoft 

Word. On numerous occasions, his guidance was helpful in improving the desired appearance of 

the body of this report and his patience in creating the tables of contents, tables, and figures. 
1

1. Introduction 

Manpower and Personnel Integration (MANPRINT) type concerns have been around for a long 

time in many fields, but not by that name. The term “MANPRINT” came into being in 1984. 

Continued systemic problems during development, or following release of the system to soldiers 

in an operational environment, led to an initial focus on correction by increasing manpower, 

recruiting more talented soldiers, and enhancing training programs. It was recognized that this 

approach, by itself, was not leading to system designs that maximized soldier performance with 

the system. Optimizing on system performance also required focusing on human requirements 

within the system. 

1.1 Development of MANPRINT Implementation 

In the 1984–1985 timeframe, the search began for people who would implement MANPRINT 

type of evaluations. Among those selected to support this initiative after systems were 

developed was a cadre of research psychologists affiliated with the U.S. Army Research Institute 

(ARI). The U.S. Army had separate Test and Evaluation (T&E) organizations—entities for 

creation of new systems. One was responsible for supporting developmental testing; the other 

was responsible for supporting operational testing. During developmental testing, problems 

were identified and presumably corrected. Our role as a Field Element was to support the Test 

and Experimentation Command (TEXCOM) in its mission of conducting operational testing. As 

individuals responsible for MANPRINT evaluations, data collected during operational testing 

provided the U.S. Army with another look at the system closer to the time when it would be 

placed in the hands of the representative soldiers. Those data served as an additional opportunity 

to determine what problems continued to exist as the system began to be used in a more missionoriented integrated form. In 1985–86, after MANPRINT had become formalized, people at 

several of the Field Elements (including ours) were sent back to the Washington area to take a 

3-week course on life-cycle management of system development. This course covered the more 

management-oriented aspects of implementing MANPRINT. As one tasked to implement 

MANPRINT, much of the information imparted seemed “good to know,” but not particularly 

useful to our mission of planning and collecting MANPRINT type data. In those early days, our 

Field Element had about ten research psychologists and so we were able to divide ourselves up to 

support TEXCOM test directorates. As structured during this period, TEXCOM was responsible 

for planning and collecting test data on systems under test during the operational test phase; the 

Operational Test and Evaluation Command (OPTEC) was responsible for performing the 

evaluation with that data. In our role of providing support for TEXCOM, it was our role to plan 

and collect the MANPRINT type data. After the 3 weeks of management-oriented training, we 
2

still had to work out the specific data collection and analysis procedures. In the 1986–1995 

timeframe, our assignments in the MANPRINT arena were managed through a form of matrix 

management. When a system was under development and some form of operational testing—

Limited User Test and Evaluation (LUTE), Initial Operational Test and Evaluation (IOTE) or a 

Follow-on Test and Evaluation (FOTE)—was being planned, our Field Element Chief 

approached us and indicated that we were being assigned to provide MANPRINT support for 

that test. While as individuals we were targeted to support systems from particular TEXCOM 

directorates, our assignments often crossed directorate lines. Once that test was completed, the 

analyst was then assigned to another system. This form of support with MANPRINT analysts 

did create some difficulties as there was always the need to become familiar with new systems; 

however, in perspective, it made us continually think about how we were going to implement the 

MANPRINT data collection and analysis effort for each specific emerging system. From that 

point on we were put in contact with the Test Officer (TO) for the system and he kept us 

apprised of test planning meetings and provided us with feedback to our emerging MANPRINT 

data collection plans. Historically (see TEXCOM memo 73-1, para 5-3a and figure 5-2, pp. 5-1 

and 5-3) (1), the role of TEXCOM was to plan and collect Level 4 data (see table 1). To support 

TEXCOM, our role was generally to prepare a MANPRINT data collection plan, collect that data 

and summarize the data in one or more forms designated for Level 4 data. 

1.2 Purpose 

The purpose of this report is to provide an overview of MANPRINT and to explain with several 

examples how MANPRINT type data is collected and to identify some common analyses used 

during conduct of T&E efforts for emerging U.S. Army systems. As will be noted in this report, 

MANPRINT evaluations of emerging systems includes identification of both Training and 

Human Factors Engineering problems. Each of these areas of interest, in themselves, is 

comprised of their own body of theory and accompanying methodological procedures. This 

document is not developed to provide the concepts or summarize the methodology they employ, 

but rather to provide specific examples of how information can be collected within these and the 

other five domains of interest discussed below. As such, the MANPRINT analyst is an 

applications generalist with backgrounds largely in psychology and human factors engineering; 

methodology used involves techniques developed in large part within the social sciences, 

statistics and operations research. These analytic techniques have been acquired throughout the 

course of study in these respective fields and are adapted to address problems evolving in 

system-specific operational and maintenance procedures. This report provides a basis for 

developing some familiarity with MANPRINT implementing procedures. It is not reasonable to 

expect that this training will make you instant experts as MANPRINT analysts. The primary 

way you develop this expertise is to use and adapt the techniques and examples presented in this 

report while conducting MANPRINT evaluations to systems on which you have been assigned to 

provide MANPRINT support. 
3

Table 1. First four levels of data. 

Level Description Possible Forms Examples of Content Disposition 

Level 1 

data: “raw 

data” 

Data in their 

original form. 

Results of field 

trials just as 

recorded. 

Complete data 

collection sheets, 

exposed camera film, 

voice recording 

tapes, original 

instrumentation, 

magnetic tape or 

printouts, original 

videotapes, filled 

questionnaires, 

interview notes. 

1. All reported target 

 presentations and 

 detection. 

2. Clock times of all 

 events. 

3. Azimuth and vertical 

 angle from each flash 

 base for each 

 flash. 

4. Recording tapes of 

 interviews. 

Accumulated 

during trials 

for processing. 

Usually discarded 

after use. Not 

ordinarily given 

to another 

agency. Not 

published. 

Level 2 

data: 

“reduced 

data” 

Data taken from 

the raw form and 

consolidated. 

Invalid or 

unnecessary data 

points deleted. 

Trials declared 

“no test” 

Deleted. 

Confirmed and 

corrected data 

collection sheets, 

film with extraneous 

footage deleted, 

corrected tapes of 

printouts, and 

original raw data 

with “no test” events 

marked out. 

1. Record of all valid 

 detections. 

2. Start and stop times 

 of all applicable 

 events. 

3. Computed impact 

 points of each round 

 flashed. 

4. Confirmed interview 

 records. 

Produced during 

processing. 

Usually discarded 

after use. Not 

published. 

Level 3 

data: 

“ordered 

data” 

Data which have 

been checked for 

accuracy and 

arranged in 

convenient order 

for handling. 

Operations limited 

to counting and 

elementary 

arithmetic. 

Spread sheets, 

tables, typed lists, 

ordered and labeled 

printouts, purified 

and ordered tape, 

edited film, edited 

magnetic tapes, 

ordered punch cards 

1. Counts of detections 

 arranged in sets 

 showing conditions 

 under which 

 detections occurred. 

2. Elapsed times by 

 type events. 

3. Impact points of 

 rounds by condition 

 under which fired. 

4. Interview comments 

 categorized by type. 

Not usually 

published but 

made available to 

analysts. Usually 

stored in 

institutional 

data banks. All 

or part may be 

published as 

supplements to 

test report. 

Level 4 

data: 

“findings” 

or 

“summary 

statistics” 

Data which have 

been summarized 

by elementary 

mathematical 

operations. 

Operations limited 

to descriptive 

summaries; no 

judgments or 

inferences. Does 

not go beyond 

what was 

observed in test. 

Tables or graphs 

showing totals, means, 

medians, modes, 

maximums, 

minimums, quartiles, 

deciles, percentiles, 

curves, or standard 

deviations. Qualitative 

data in form of lists, 

histographs, counts by 

type, or summary 

statements. 

1. Percentage of 

 presentations 

 detected. 

2. Mean elapsed times. 

3. Calculated probable 

 errors about the 

 centers of impact or 

 conditions. 

4. Bar graph showing 

 relative frequency 

 of each category of 

 comment. 

Published as the 

basic factual 

findings of test 

report. 
4

2. Method 

Following the beginning of the U.S. Army’s Integrated Test and Evaluation (ITE) initiative 

around 1996, the author was assigned to provide MANPRINT support to one of the pilot 

systems—the All-Source Analysis System (ASAS) Block 2 (Modifications). With this 

assignment, greater continuity came in the system evaluation process as support shifted to 

different sub-systems of ASAS. From that point until the Block 2 IOTE in 2005, with one 

exception, MANPRINT support was for ASAS sub-systems. 

In 1997, the Intelligence Directorate at Fort Huachuca contacted the Fort Hood Field Element 

and requested guidance on how to implement MANPRINT. With the sizable number of 

MANPRINT support efforts that had been conducted up through 1996, the immediate question 

asked was why those evaluation efforts could not be used as the basis for the guidance they 

sought. The answer derived largely from the distinction between reports that deal with the 

process—the specific ways the data are obtained—and those that focus on product—the 

identification of specific MANPRINT problems that follow from an assessment of findings from 

a body of reports that had addressed MANPRINT problems. As noted above, our early 

affiliation with TEXCOM led to more process-oriented expertise. For the most part, existing 

System Evaluation Reports (SER) detailed methodology about how those results were 

obtained—the process—was often lacking or presented with a “light brush.” Study of the 

findings presented in the more product-assessment-oriented reports can lead to identification of 

information types that need to be collected, but generally do not provide a clear picture of the 

instruments’ structure or how they are used to collect that information. While procedures 

outlined in this tutorial are based on a sizable number of emerging systems that were under test, 

in the author’s experience, there did not seem to be a single set of procedures for providing 

MANPRINT support. In some cases, the specific examples presented are system-specific (e.g., 

specific system operational and maintenance tasks) and are designed more to give the evolving 

MANPRINT analyst a feel for the type and level of detail reflected in the systems under study; in 

other cases, the information collected is generic and has direct applicability to multiple systems. 

It should be understood that the examples provided are just that—examples—and judgment must 

always be exercised. 

This section provides an overview (or characterization) of MANPRINT. Also presented are 

MANPRINT models for evaluation and some recommended forms that can be used as beginning 

examples with each of those process-oriented models. Appendices provide some alternate data 

collection forms, cite alternative domain-specific detailed data collection areas, presents some of 

the more administrative procedures directly supporting the data collection implementation and 

provides a sample analysis that has been particularly useful in addressing soldier samplerepresentativeness. While the MANPRINT Analyst is involved in the data collection effort, it 
5

should become clear that this person will not generally provide the same level of support for each 

domain (see AR 602-2, table 3-1, p 11) (2). That table indicates that the major proponency for 

data collection in each of the domains is assigned according to the Acquisition Category (ACAT) 

and whether the system under test is an Integrated Concept Team (ICT) or an Integrated Product 

Team (IPT) test. For the most part, the primary responsibility of the MANPRINT analyst to 

whom this tutorial is directed is in the Human Factors Engineering domain. For the other 

domains, his responsibility is more that of a data collector and observer who reports problems 

that appear to be attributed to a domain. In those cases, the major proponent's responsibility is to 

review and validate this data and include it in his assessment with his analyses In those cases, 

the pieces of information reported by the MANPRINT Analyst are combined with more detailed 

data collected and, according to the “intersect theory of assessment” (3), serve as confirmation 

(or contradiction) to the proponent's data collection and analysis efforts. 

This tutorial was originally designed to be presented to groups of individuals who desired 

“heads-up” training as MANPRINT analysts; however, in this presentation, the material has been 

restructured to be better suited for individual review and orientation. This restructuring was 

completed with the recognition that those who might serve as facilitators for training several 

individuals probably have a level of expertise in MANPRINT evaluation that they would prefer 

to use and in a manner more suited to their experiences. Like the proverbial “blind men trying to 

describe an elephant,” different people will have differing opinions. 

2.1 Characterization of MANPRINT 

MANPRINT has been defined as Manpower and Personnel Integration. At present, MANPRINT 

is characterized by seven domains, or areas of interest. Discussion in this section provides a 

more general description of matters and issues addressed in each of these areas of interest. 

2.1.1 Manpower Domain 

Within this area of interest, the basic question addressed is: Are there enough soldiers to operate 

and maintain the system? Factors addressed when considering this area are the workload and 

types of job stress associated with performing system specific critical tasks. Other sources of 

problems that may be related to the Manpower Domain are based on soldiers' judgments about 

the probable causes for reported problems. 

The definition of manpower in terms of the Training and Doctrine Command (TRADOC) 

Systems Manager (TSM) and Combat Developer (CD) is to provide the guidelines of how many 

people the system needs to be operated and maintained. MANPRINT analysts don’t actually set 

any requirements. The MANPRINT analyst looks at what the operators and maintainers’ 

performance evidences (usually through reported problems and performance deficiencies) and 

then makes a judgment about whether the U.S. Army should rethink how many soldiers they 

need to do the job—operate and maintain the system. That information is provided to the U.S. 

Army Test and Evaluation Command (ATEC) with supporting rationale. 
6

2.1.2 Personnel Domain 

Within this area of interest, there are three major questions addressed: (1) Do we have the right 

type of soldiers “manning” the system (MOS/skill level)?; (2) Are soldiers participating in 

developmental tests among the best of those who will operate and maintain the system? 

Comments of those soldiers are needed to identify problems with the system that may be 

especially troublesome for the aver age soldier and to motivate systemic changes in procedures 

and/or equipment before it is placed in the hands of the representative soldier; and (3) Are the 

test players in the Operational Test representative of the target audience for whom the system is 

intended? The MOS selected for the system under study are generally those who had been 

operating and maintaining the predecessor system. This domain addresses whether those 

soldiers have the appropriate abilities and the right skills. When soldiers report a particular 

problem may have occurred because the “right” type of soldiers are not “manning” the system, 

this suggests that the source of the problem may be attributed to the Personnel Domain (4). In 

developmental testing, it is best to use “golden crews.” In earlier tests and evaluations, it had 

been noted that these superior crews were sometimes used in operational testing and had the 

consequences of making the system appear better than it turned out to be when it was placed in 

the hands of the more representative soldier. Those are the people you want in developmental 

testing because they will identify the problems that exist in the system and are likely to cause 

problems for the typical soldiers who will operate and maintain the system. In an operational 

test, you want representative (“typical”) soldiers. To make that kind of judgment, procedures for 

determining what representative means must be developed. Addressing the latter two major 

questions involves exploring the demographic background characteristics and ability measures 

like Armed Services Vocational Aptitude Battery (ASVAB) scores of the selected soldiers and 

comparing them with those exhibited by the ones who operated and maintained the predecessor 

system. 

Major resources used to make comparisons between the test sample selected and the population 

(target audience) for whom the intended system is being designed involves contacting a 

Department of Defense repository of this information. Previous efforts have indicated that the 

Defense Manpower Data Center (DMDC) in Monterey, CA, is a responsive source. Following 

September 11, DMDC reorganized and new emerging procedures have been established for 

obtaining demographic data and ASVAB scores for active duty enlisted soldiers and 

demographic data for active duty officers. 
7

Details for contacting DMDC include the following information: 

• Initial POC: Michelle Rudolph, Branch Chief for Personnel and Manpower 

• Phone: (831) 583-2400; DSN 878-2951; FAX (831) 583-2340 

• Address: DMDC, DOD Center, Monterey Bay, Seaside, CA 93955-6771 

• Establish Request: Data Request System (DRS): https://www.dmdc.osd.mil/drs/ 

• Enter: DMDC Assigned Name and Password 

• Enter: Request Title 

• Choose Subject: e.g., Accessions 

• Check Groups: e.g., Active Duty Military and MEPCOM 

• Enter Detailed Description: “Support MANPRINT Assessment in Personnel Domain 

for the ____________ System” 

 -Reference memorandum documenting request (sample presented in appendix A). 

 -When contacted by DMDC analyst, reference that analyst in the memo ATTN line. 

Appendix B provides a sample of how information provided by DMDC can be used to address 

the sample representativeness of the soldiers participating in an operational test. 

2.1.3 Training Domain 

Within this area of interest, the major question addressed is the adequacy of the training provided 

to soldiers to operate and maintain the system. The major parameters addressed in a training 

evaluation include: 

1. What is being trained 

a. Individual tasks 

b. Collective tasks 

2. What is the type of training evaluation 

a. Process 

 i. Training environment adequacy 

 ii. Instructional/learning difficulties
8

 b. Product 

 i. Performance (critical task time and error) 

 ii. Soldier problem reports 

3. When does the training evaluation occur 

 a. Training Classroom 

 b. Collective Training environment (prior to Record Test) 

 c. Mission Enclave (at end of Record Test) 

The evaluation focuses on both individual tasks--more generally the procedures for manipulation 

of system software—and collective tasks—more generally the mission-oriented tasks that are 

affected by performing sequences of individual tasks in specific orders. In teaching each of these 

types of tasks, process and product evaluations are conceptually possible. In the process 

evaluation, there is generally an examination of the adequacy of the training environment and 

review of instructional and learning difficulties. In the product evaluation, tasks’ performance 

(time and error scores) are reviewed to permit a judgment about deficiencies in the method of 

instruction or specific reasons soldiers experience learning difficulties. In the training 

environments (classroom and mission facilities), product evaluation may include hands-on 

exercises and/or performance-based written tests. During the test where soldiers are performing 

in a mission-oriented environment, they are asked to make judgments about their performance 

and indicate causes for performance difficulties. Those causes can be attributed to one or more 

domains, and problems experienced in one domain may be caused by problems in another 

domain. 

2.1.4 Human Factors Engineering Domain (5–7) 

Within this area of interest, three basic questions about the operational and maintenance 

procedures asked of the emerging system include: (1) Is the equipment/hardware difficult or 

complicated to use by the soldier?; (2) Are there problems with the software used in the system?; 

and (3) Are there problems with the procedures? 

Evaluation within this domain focuses on hardware/software interface problems used to 

complete system-required individual and collective tasks. Method of evaluation within this 

domain generally varies from mapping soldiers' description of problems with different critical 

tasks, presented as apparent causes for problems, to use of a whole host of different surveys. 
9

Those surveys may focus on equipment components and their physical characteristics, such as 

those included in appendices C (8) and D (9). They may also focus on menus and pull-down 

menus, procedures, function-specific software actions, or specific soldier physical dimensions. 

In those surveys, judgments about these areas of focus may ask about their level of adequacy or 

effectiveness. Where judgments are negative, soldiers are encouraged to describe the reasons on 

the survey form or in scheduled follow-up interviews. Many of the systems used today have 

computer equipment and software to complete MOS-based tasks. In some sense, completing 

those tasks as they were originally trained--in a “manual” (or analogue) mode (without a 

computer) involved different sets of activities (procedures) that often do not parallel those that 

are required when those tasks are completed with a series of computer manipulations using the 

software. This lack of parallelism can create problems in both learning and executing the 

intended tasks so that they are consistent with how the soldier conceptually thinks about 

completing the tasks to perform mission objectives. Many times, the procedures for completing 

the mission without automation are quite different from those used when there is computer 

augmentation. 

2.1.5 Health Hazards Domain (10, 11) 

Within this area of interest, the major question asked is: Does operation or maintenance of the 

system effect soldier safety? For both this domain and the next, soldiers' reports of problems 

operating and maintaining the equipment and Test Incident Reports (TIRs) serve as the two 

primary sources of data collected by the MANPRINT Analyst that tend to serve as confirmation 

of known problems or initial identification of problems needing further exploration. 

2.1.6 System Safety Domain (10, 11) 

Within this area of interest, the major question asked is: Does operation or maintenance of the 

system affect safety of the system? 

2.1.7 Soldier Survivability Domain 

Within this area of interest, the MANPRINT analyst focuses on soldiers’ operational and 

maintenance problems while wearing Mission Oriented Protective Posture (MOPP) gear. Four 

additional areas of interest within this domain include: (1) Nuclear Survivability Testing; (2) 

Biological and Chemical Survivability Testing; (3) Information Warfare Testing; and (4) 

Electronic Warfare Testing. As the MANPRINT analyst generally does not have either the 

specific skills or equipment to address these latter areas of interest, their assessment usually is 

coordinated with the U.S. Army Research Laboratory (ARL) Survivability/Lethality Analysis 

Directorate (SLAD) and ATEC. 
10

2.2 MANPRINT Models for Evaluation 

Based on review of the MANPRINT evaluation data collected primarily by the author for several 

systems (mostly in the 1986–1995 period), it seemed that the procedures used could generally be 

characterized as one of four types. Table 2 shows these four types (models) as a consequence of 

two factors: (1) Availability of System Critical Tasks and (2) Level of Verbal Skills of those for 

whom the system was designed. Superscripts in table 3 are footnote references for sample 

systems shown in table 2 that identify general planning factors for conducting the MANPRINT 

evaluation; for the Heavy Equipment Transporter System (HETS) IOTE the superscript also 

indicates the multiplier effect of providing additional support to the MANPRINT data collection 

effort—“what you can do with the resources you’ve got.” In most cases, entries in this matrix 

are the names of the systems under test for which the indicated MANPRINT Evaluation model 

was used by the author. While these factors are helpful to provide initial direction on which 

MANPRINT model to use in an evaluation effort, it is important to remember that “every test is 

different.” 

Table 2. Models for MANPRINT evaluation (but “every test is different”). 

High Verbal Skills Low Verbal Skills 

Critical Tasks 

Available 

Model 1 

Various ASAS LUTE and Block 2 

IOTEa

 (16) 

Model 2 HETS IOTEb

 (13)

Enhanced Position Location Reporting 

System (EPLRS) IOTEc

 (17)

Critical Tasks 

NOT Available 

Model 3 

War Fighting Rapid Acquisition 

Program (WRAP): ASASd

 (12) 

Model 4 M1A2e

 (14)

Secure Mobile Anti-Jam Reliable 

Tactical Terminal (SMART-T)f

 (15)

Note: Footnotes a–f are explained in table 3. 

Citations for the systems listed in table 2 are presented in the references section (12–17). Data 

collection procedures employed for the MANPRINT Evaluations with each model follow from 

consideration of the characteristics defining the model and the resources/conditions existing for 

the system being evaluated. The body of this report explains the more salient features of the 

different data collection procedures for each model. In addition to this discussion, several 

appendices are used to present alternative forms—some requesting information across 

MANPRINT Domains, others requesting more detailed type of information from specific 

domains. While MANPRINT Analysis Results are usually presented by the MANPRINT 

Domain, events leading up to the Record Test have a logical sequence and tend to direct the 

order in which data collection proceeds. As the purpose of this paper is to guide the planning for 

implementing MANPRINT data collection, the focus of this tutorial is on sample forms used 

within each model in the order that they are used. 
11

Table 3. Footnotes referenced in table 2. 

Footnote Model Who Test Respondents 

Test 

Parameters 

MANPRINT Support 

Team 

MAN

a 1 OHH 

Various ASAS LUTE and Block 2 IOTE 

40 (half being data collectors) 5–6 days 1 GS12 (MGR) 

Develop DDesign DCollect anProvide aWrite rep

b 2 OHH HETS-IOTE 50 (half being 

data collectors) 

6 months 

1 GS12 (MGR) 

1 contractor w/sys 

knowledge and data 

processing skills 

1 Temp w/yrs of test 

experience 

Develop DDesign DCollect anData AnaWrite Rep

c 2 OHH EPLRS-IOTE 

200 soldiers, variety of MOSs and skill levels 

1 month 

3 phases 

1 GS12 (MGR) 

3 GS5 temps 

1 experienced 

4 military 

Develop DDesign Dcollect anProvide O Level 

d 3 OHH WRAP: ASAS 10 soldiers 3 days 1 GS12 

Develop DDesign DCollect anReport fin

e 4 NDS M1A2-IOTE 80 armor MOS 

4 months 

2 phases 

1 GS13 (MGR) 

1 Temp (CHF) 

Senior retired NCO w/sys 

experience, 4 temps 

Ret senior NCOs 

Background in system 

TEXCOM did data entry 

Develop DDesign DCollect anProvide O Level 

f 4 OHH SMART-T 30 soldiers 3 days 1 GS12 

Develop DDesign DCollect anreport fin
12

2.3 Resources and Data Collection Procedures for Conducting MANPRINT Evaluations 

With Four Different Models 

The MANPRINT Evaluation models have been presented in table 2. Entries within this table 

are, with one exception, systems for which the author was requested to provide MANPRINT 

support. Models 2 and 4 were used to support MANPRINT evaluations for two systems each. 

In each case, systems supported by these evaluation models have similarities, but differ in the 

resources available and (for model 2) the conditions of testing (see table 3). 

2.3.1 Use of Model 1 for MANPRINT Evaluation of the All-Source Analysis System 

(ASAS) 

The model designated as model 1 appears to work well with soldiers who have high verbal 

skills—like those who operate and maintain the All Source and Analysis System (ASAS)—when 

individual and collective tasks have been identified. When approaching a new system to conduct 

a MANPRINT evaluation, it is important to make a judgment about the verbal skill of the 

soldiers with whom you will be dealing and whether or not system trainers can provide you with 

a list of critical tasks used by the system. 

2.3.1.1 Resources Used for MANPRINT Evaluation of ASAS. There were about 40 system 

participants—half of whom were data collectors—for a 5–6-day test. Except for the New 

Equipment Test Team (NETT) instructors, all respondents were soldiers assigned a relevant 

Military Intelligence (MI) Military Occupational Specialty (MOS). The MANPRINT support 

team—the resources available to conduct the MANPRINT evaluation—involved just one 

MANPRINT analyst. What was one MANPRINT analyst able to do? He drew up the data 

collection plans, designed the data collection instruments, collected and processed the data, 

analyzed the data and wrote draft MANPRINT input for the System Analysis Report (SAR) and 

System Evaluation Report (SER). 

2.3.1.2 Data Collection Procedures for MANPRINT Evaluation of ASAS. In implementing the 

MANPRINT evaluation using Model 1, it was frequently helpful to develop an Event Design 

Plan (EDP) to guide the progression of the evaluation events. An example of this plan is shown 

in table 4. That table was developed specifically to support an assessment for an ASAS test. 

Comparable tables were developed for each of the ASAS tests supported (LUTEs and the Block 

2 IOTE). It provided a simple way to guide what data is needed, when to collect it and from 

whom to collect it. Samples of the forms used are presented in the same sequence that they are 

presented in this EDP. The first column in this EDP is the Form Name. File names identify the 

particular instrument. In this test, there were several functionalities; those are listed in the 

second column. The third column refers to the time when you use each form. The fourth 

column indicates who are to provide responses to items on this form, and the last column is the 

purpose of using the instruments. As already noted the purpose of this paper is to direct the 

planning for implementing MANPRINT data collection, the focus of this presentation is on 

sample forms used within each model in the order that they are used.
13

Table 4. Event design plan (EDP) for the all-source analysis system (ASAS) block 2 Remote Work Station (RWS) limited user test an

collection plan. 

Form Name Critical Task List Time Used Respondents 

Training Evaluation Debrief  (TADB_RWS.WP6)  (TADB_AMR.WP6) 

(TADB_EAR.WP6) (TADB_SAR.WP6) 

Remote Work Station-RWS Asset Management ELINT Analysis System Administration 

At end of each week for each instructional block for each set of students 

Analysts Data Collectors Instructors 

Identrain

ASAS Training Process Questionnaire 

(AS_TNG_Q.WP6) 

All Critical Tasks At end of each instructional block 

for each set of students Used with (after) completing “Training Assessment Debrief” 

Analysts Data Collectors 

Dociden

Demographic Questionnaire 

(PLYSDEMR.WP6) — 

At beginning of training Analysts 

Data Collectors 

Docand 

MANPRINT Debrief 

(DB_RWS.WP6) (DB_AMR.WP6) (DB_EAR.WP6) (DB_SAR.WP6) 

Remote Work Station-RWS Asset Management ELINT Analysis System Administration 

At end of each test day (Pilot & Record Test) 

Analysts Data Collectors Doc

and prob

MANPRINT Evaluation 

(MP_ASSMR.WP6) 

— 

Used in conjunction with (after) “MANPRINT Debrief”. One form completed for each problem report--at end of each test day (Pilot & Record Test) 

Analysts Data Collectors 

Provcomtaskdom
14

Table 4. Event design plan (EDP) for the all-source analysis system (ASAS) block 2 Remote Work Station (RWS) limited user test an

collection plan (continued). 

Form Name Critical Task List Time Used Respondents 

ASAS Human Factors Engineering (General Software Functioning) questionnaire (OEC provided--61 items) (SSHFEQ.WP6) 

— 

Final Debrief--day after last Record Test day 

Analysts Data Collectors 

Assprob

ASAS Human Factors Engineering (FunctionSpecific) Questionnaire for each ASAS Function 

RWSHFEFunctionsSpecificquestions.doc AssetManagementHFEFunctionsSpecificquestions.doc ELINTHFEFunctionsSpecificquestions.doc COMINTHFEFunctionsSpecificquestions.doc SystemAdministrationHFEFunctionsSpecificquestions.doc 

Final Debrief--day after last Record Test day 

Analysts Data Collectors 

Finasyst

ASAS Interoperability Problems 

interop3-ace_iote_blk2.doc Final Debrief--day after last Record 

Test day 

Analysts Data Collectors 

DocrecemesBFA

Final Debrief ASAS Single Source Operator /Analyst Questionnaire (SSOAR.WP6) 

— 

Final Debrief--day after last Record Test day 

Analysts Fina

syst

Final Debrief ASAS Single Source Data Collector Questionnaire 

(SSDCR.WP6) 

— 

Final Debrief--day after last Record Test Day 

Data Collectors Fina

syst
15

2.3.1.2.1 Training Evaluation Debrief (table 5). The instruments in the first row, first column of 

table 4 are for part of the training process evaluation in the four ASAS functionalities being 

addressed. The sample of this form references tasks for the Remote Work Station (RWS) 

Functionality. They are recommended for use at the end of each week for an instructional block 

of training for each set of students. The MANPRINT Analyst needs to coordinate with the 

NETT leader to determine, when exactly, would be the best time to come in and survey these 

soldiers and to indicate those tasks that had been trained during that week. Conducting surveys, 

at least on a weekly basis, is desirable so that soldiers who have a problem can provide relatively 

fresh knowledge about it; otherwise new problems tend to merge with the old and then tend to be 

forgotten. 

2.3.1.2.2 Training Process Questionnaire. The second row, first column of table 4 lists an 

evaluation instrument designed to address the training process. This form specifically addresses 

different aspects of the training environment. It has been generally an instrument used at the end 

of each instructional block for NETT (individual tasks) type of training—in contrast to weekly as 

with the training evaluation debriefs. Potentially, analysts, data collectors and instructors can all 

provide information about the training evaluation at the individual critical task level. The 

analysts and data collectors are probably the ones who can provide the most valid information 

about the training environment. They can document problems that arose during training. 

In appendix F, another version of this training questionnaire used during the Maneuver Control 

System (MCS) IOTE is presented (18). 

2.3.1.2.3 Demographic Questionnaire. It’s not clear that there’s any best time to administer the 

demographic questionnaire—see row 3, column 1 of the first page of table 4. Part of the reason 

for administering it at the beginning of training derives from the understanding that, in 

implementing MANPRINT, there is a lot of “paperwork” to be completed. If, without 

jeopardizing your data collection effort, forms and surveys are distributed throughout the entire 

data collection effort to minimize the amount of “pain” the soldiers have to experience and make 

it easier for the Test Officer to accommodate the data collection intrusions throughout the test. 
16

CLASSROOM 

NETT TRAINING EVALUATION DEBRIEF 

ASAS BLOCK 2 OPERATIONAL TEST 

RWS V6.3 

Name ____________________________________________________ PIN no. ___________ 

 (last 4 SSN) 

Please Circle Equipment Used: HCU VCU Date_____________ 

MOS___________ Rank___________ 

Instructions

Respond for each task: 

1. Most listed TASKS have performance steps (in parentheses following TASK NAME) you could perform in 

completing the TASK. These steps are shown to help you focus on the aspect(s) of the task that may have 

been difficult (to learn or requiring instruction modification). 

2. Circle Y in INSTRUCTIONAL DIFFICULTIES column (Analysts) if there appeared to be difficulties in 

training this task by the instructor. If there are no problems, leave uncircled. 

3. Circle Y in the LEARNING DIFFICULTIES column if you (the Analyst being trained) experienced some 

difficulty in learning this task. If there are no problems, leave uncircled. 

4. If you think you know the cause of the DIFFICULTY (Instructional or Learning), use the back of this sheet or 

attached sheet to respond. Indicate: (1)TASK NO; (2)Particular performance step creating difficulty; and (3) 

your response. Among the many possible causes for the difficulty might have been: (1) Software/hardware 

problems, (2) instructor teaching technique/preparation, (3) lack of repetition or PEs, (4) classroom setup, 

noise or distractions, (5) task complexity, (6) manuals, ... Others may exist. 
17

Table 5. Training evaluation debrief data collection. 

Task No. Task Name 

Instructional 

Difficulties 

Learning 

Difficulties 

RWS-0001 Assemble the RWS Y Y 

RWS-0002 Configure the RWS for Operations 

[A)Boot RWS, B)As Standalone w/wo TOCBS, 

C)/As Master w/wo TOCBS, D)As Client w/wo 

TOCBS, D)As Client w/wo TOCBS] 

Y Y 

RWS-0003 Initialize the ASAS-RWS Software 

[SELECT: A)Functional Identities, B)Time Zones, 

C)Start Option Radio Button, D)Configuration, 

E)Initialization Button] 

Y Y 

RWS-0004 Log on to The RWS System Y Y 

RWS-0005 Perform Shift Change Procedures Y Y 

RWS-0006 Terminate System Operations 

[A)Stop-A/Control Break] 

Y Y 

RWS-0007 Disassemble the RWS Y Y 

RWS-0009 Perform Desktop Tools Menu Operations 

[A)Set Display Filters, B)Enable/Disable Alarms, 

C)Select Profiles, D)Mount Floppy Disk/CD 

ROM/Optical Disk, E)Format Floppy Disk/Optical 

Disk] 

Y Y 

RWS-0010 Perform Printer Management and Hardcopy Services 

[A)Window, B)Screen, C)Window to file, D)Remove 

print job from queue, E)Administer print labels, 

F)Display printer status, G)Select user default printer, 

H)Add/Delete printer, I)Change IP Address, J)Printer 

Diagnostics] 

Y Y 

RWS-0011 Perform File Browser Functions 

[A)Delete/Move/Copy/Change Permissions, 

B)Change Access Control Settings, C)Access Files in 

SunPCi Environment] 

Y Y 

RWS-0012 Save Screens or Windows to Files Y Y 

RWS-0013 Configure ASAS-RWS Nodes 

 A)Add/Delete/Modify Node Information (Intel Ops 

State), B)Allow/Prohibit Client Node Host Automatic 

Processes 

Y 

Y 

RWS-0016 Select Country Codes 

[A)Search/Set Preferences] 

Y Y 

RWS-0021 Use COE Office Products 

[A)Start/Shut Down Sun PCi Environment, b)Access 

Microsoft Word/Excel/PowerPoint 

Y Y 

RWS-0022 Collaborate Using Sunforum 

[A)Launch SunForum, B)Connect to Work 

Station/Send Files/Chat, C)Share System Windows, 

D)Use Whiteboard] 

Y Y 

RWS-0023 Change Nickname/Codeword 

Y Y 

RWS-0025 Perform On-Line Operations 

[A)Access WWW Browser, B)Find and Return to 

Pages, C)Open New Navigator 

Window/Display Content, D)Create Simultaneous 

Network Connections/New Page, E)Search for 

Information] 

Y Y 
18

Table 5. Training evaluation debrief data collection (continued).

Task No. Task Name 

Instructional 

Difficulties 

Learning 

Difficulties 

RWS-0100 Perform Security Audit Trail(SAT) 

[A)Set Filters, B)Collect SAT Data, 

C)Generate/Delete/Print SAT Reports, 

D)Archive/Purge/Restore Files] 

Y Y 

RWS-0101 Change System Security Parameters 

[A)High Water Mark(HWM), B)HWM with 

Classified Help, C)Shift Change Maximum 

Time/Screen Saver Time] 

Y Y 

RWS-0102 Maintain User Accounts and Profiles 

[A)Create/Modify/Delete/Disable, B)Edit Account 

Profile Management(APM), C)Assign/Change 

Password] 

Y Y 

RWS-0103 Control System Process Operations 

[A)Monitor Process Status, B)Disable/Enable/Restart 

Processes] 

Y Y 

RWS-0200 Perform Comms Message Processor(CMP)Functions 

[A)Designate/Start/Monitor a CMP] Y Y 

RWS-0201 Perform Messaging Using CMP 

[A)Launch Messaging Tools, B)Confirm 

Connections, C)Set Message Wrapper Defaults, 

D)Send/Edit/Review/Archive a Message with CMP, 

E)Select/Use Message Filter, F)Use Message 

Handling Table] 

Y Y 

RWS-0202 Produce Enemy Interoperability Messages(EIM) 

[A)Configure EIM for Automatic/Manual Release, 

B)Modify/Disable a Destination for EIM, C)Generate 

EIM from Entity Operations, D)Generate an S507 

Resources Logistics Message] 

Y Y 

RWS-0203 Control Message Release Authority(MRA) 

[A)Enable/Disable MRA, B)Process a Message] Y Y 

RWS-0204 Prepare Input/Output Media 

[A)Load from Input Media, B)Save to Output Media] Y Y 

RWS-0205 Send a Message Manually 

[A)Prepare/Send(Release), B)Add Addressee] Y Y 

RWS-0206 Process an Inbound Message 

[A)Interactive Message Parsing, B)Text Message 

Processing, C)Message Criteria Alert Processing] 

Y Y 

RWS-0207 Manage Files Using X-FTP 

[A)Access X-FTP, B)Connect/Disconnect to 

Destination, C)File Management] 

Y Y 

RWS-0208 Perform Address Group Maintenance 

[A)Create/Delete/Maintain] 

Y Y 

RWS-0210 Perform Analyst Mail Functions 

[A)Create Address Book from TOCBS, B)Send Mail] Y Y 

RWS-0211 Process Anomalous Messages 

[A)Add Classified Alias for Single Message/Message 

Processing, B)Recheck/Delete] 

Y Y 
19

Table 5. Training evaluation debrief data collection (continued).

Task No. Task Name 

Instructional 

Difficulties 

Learning 

Difficulties 

RWS-0212 Communicate Using Chatter 

[A)Initiate/Answer Chatter Call] 

Y Y 

RWS-0213 Perform Message Journal Review Y Y 

RWS-0214 Perform Address Maintenance 

[A)Create/Modify/Add Address Information, 

B)Change Data Transfer Method Preferences, 

C)Make Global Changes to System Types, D)Use 

Destination Addressing] 

Y Y 

RWS-0301 Monitor The Network Operational Status 

[A)Access Tool, B)Map Network/Save, C)Modify 

Network Node Attributes/Map Display] 

Y Y 

RWS-0302 Access Remote Hosts Y Y 

RWS-0305 Modify Network Configuration 

[A)Network Configuration Management Functions] Y Y 

RWS-0400 Create Alert Criteria Development Sets 

[A)Criteria Record, B)Activate/Deactivate Criteria 

Set/Record, C)Update Criteria Record, D)Set Analyst 

Preferences and Filters (APAF)] 

Y Y 

RWS-0401 Create Message Criteria Development Sets Using 

The RWS 

[A)Create Standing Request for Information(SRI), 

B)Stop/Start Message Criteria Processing, C)Process 

a SRI Alert] 

Y Y 

RWS-0402 Perform Alert Operations--Data Criteria Alerts(DCA) 

[A)Acknowledge a Data Criteria Alert, B)Plot an 

Entity from Data Criteria Alerts Window, 

C)Generate/Transmit DCA] 

Y Y 

RWS-0404 Post and Manage Imagery Related Intel Products 

Using ELT/4000 

[A)Create Graphic Image, B)View/Manipulate Image 

Using ELT/4000, C)Download Secondary Imagery, 

D)Display Camera Icon, E)Register Image to Map, 

F)Import a Textual INTSUM, G)Display Image Icon 

and Its Footprint to a Map, H)Sort Image File, I)Set 

View for Image File, J)Delete Image File, 

K)Disseminate SID via E-Mail] 

Y Y 

RWS-0405 Process Candidates for Specific Correlation 

[A)Open Specific Correlation Interactive Window, 

B)Set Display of Records, C)Insert Records into 

ASCDB, D)Retrieve Candidates for Correlation, 

E)Forward a Record for Alternate Processing, 

F)Remove Candidate from Display, G)Combine 

Records] 

Y Y 

RWS-0406 Develop the Collection Plan 

[A)Create, B)Send/Receive Plan/Synchronization 

Matrix, C)Share Collection Plan in Multinode 

Environment] 

Y Y 

RWS-0407 Develop the Intelligence Synchronization 

Matrix(SM) 

[A)Create SM, B)Generate/Transmit Multiple Asset 

Tasking Message(MATM)] 

Y Y 
20

Table 5. Training evaluation debrief data collection (continued).

Task No. Task Name 

Instructional 

Difficulties 

Learning 

Difficulties 

RWS-0408 Develop a Doctrinal Template(DT) 

[A)Take Entities on the Map, B)Plot DT to Map, 

C)Create Situational Template from DT/Observations 

on Map and Both Sources, D)Create an Event and 

Decision Support Template] 

Y Y 

RWS-0410 Perform ASAS Overlay Operations 

[A)Create/Load/Open/Send/Export/Import Overlays, 

B)Make Overlay Available for MCO] 

Y Y 

RWS-0411 Perform CTP/Overlay Operations 

[A)Start the Common Tactical Picture Application, 

B)Create a New Chart Tab, C)Set a Chart Tab to 

Archive in Overlay Explorer, D)Create Plan/Overlay 

in the Overlay Explorer, E)Add Objects to 

Plan/Overlay, F)Save/Remove/Delete Overlay, 

G)Create a SA Overlay in Overlay Explorer, 

H)Create SA Overlay for Enemy/Friendly Unit 

Information in Overlay Explorer, I)Edit CP Filter, 

J)Save Changes in Overlay Explorer, K)Open Palette, 

L)Select Palette, M)Create New Palette, N)Access 

Symbol's Definition, 

Y Y 

O)Select Multi Point Symbols, P)Create/Edit 

/Save/Delete Symbol, Q)Plot Symbols to Map from 

Milsym Manager Window, R)Move Single/Multiple 

Symbols, S)Edit Objects on Map, T)Close CTP 

Application] 

 

RWS-0412 Manipulate Battlefield Geometry 

[A)Create Friendly/Enemy Control Measure, B)Edit 

Control Measure, C)Send S201 Geometry Message] 

Y Y 

RWS-0413 Maintain Geographical Areas 

[A)Create/Edit/Delete] 

Y Y 

RWS-0500 Perform Entity Operations 

[A)Select Entities from Map 

Window(Manually/Using Criteria), B)Remove Entity 

Graphics, C)Restore Entity Location, D)Center Map, 

E)Query Database, 

F)Update/Insert/Edit/Delete/Associate/Combine 

Entities in Database, G)Generate Messages] 

Y Y 

RWS-0501 Display Entity History Y Y 

RWS-0502 Perform Map Operations 

[A)Launch JMTK, B)Save Map Area, C)Recall Map 

Area, D)Create JMTK Snapshot, E)Change Map 

Properties, F)Change Map Properties, G)Display 

Map Contours(Terrain/Bottom), H)Adjust Map 

Display Intensity, I)Recenter Map Display, J)Display 

Center Marker, K)Change Map Display 

Scale(Zooming), L)Display Grid Lines on Map 

Display, M)Load Map Data from CD ROM, 

N)Perform Quick Plot Operations, O)Perform 

Coordinate Conversion, P)Set Display Preferences] 

Y Y 
21

Table 5. Training evaluation debrief data collection (continued).

Task No. Task Name 

Instructional 

Difficulties 

Learning 

Difficulties 

RWS-0600 Execute Database Operations 

[A)Perform Forms-Based Database Query, 

B)Perform Expert Query(SQL-Based), C)Save 

Query, D)Save Results of Query, E)Load Query, 

F)Print Matrix Results of Query, G)Print Saved 

Query/Results, H)Plot Query Results, I)Center Map 

to Selected Query Results, J)Update Record, 

K)Customize Display of Query Results, L)Convert 

Coordinates, M)Set Default Map for Map Operations 

from Query, N)Create Entity(Long/Short Form), 

O)Create Damage Assessment/Control Measures, 

P)Access Geographical Areas and Query, Q)Retrieve 

Message for Queried Item, R)Generate Messages] 

Y Y 

RWS-0601 Create Database Entities 

Y Y 

RWS-0603 Combine Entities 

[A)Combined Plotted Entities from Map, B)Change 

Views/Fields] 

Y Y 

RWS-0604 Maintain Enemy Order of Battle Tables 

[CREATE: A)New Facility Types, B)Facility Alias, 

C)Equipment Alias, D)New Equipment Type, E)New 

Force Type, F)New Major Branch Type, G)New Unit 

Identification Alias, H)New Unit Number Alias, 

I)New Organization Type, J)New Echelon Type, 

K)New Functional Role Type, L)New Radio Type, 

M)New Radio Alias, N)New Radar Type, O)Radar 

Alias, P)Select Classification Abbreviation] 

Y Y 

RWS-0605 Perform Parser Table Maintenance 

[MAINTAIN TABLES: A)Readdress, B)Auto 

Discard, C)Inbound Routing, D)Auto Forward 

Criteria, E)Auto Forward List] 

Y Y 

RWS-0606 Associate Entities 

Y Y 

RWS-0607 Set Criteria for Redundancy Checking of 

Incoming Messages 

[CREATE:A)New Set of Search Parameters for 

Redundancy Checking, B)New Record for 

Redundancy Checking, C)New Set of Rules for 

Specific Correlation, D)New Combination Methods 

Set/Record, ACTIVATE: E)Set of Search 

Parameters, F)Set of Specific Correlation Rules, 

G)New Combination Methods Set, 

H)Change/Deactivate Combination Methods] 

Y Y 

RWS-0608 Maintain Country Code Tables 

[A)Search/Add/Modify Country Code] 

Y Y 

RWS-0609 Query the Database Via WWW Y Y 

RWS-0610 Navigate the Table Maintenance User Interface Y Y 

RWS-0612 Maintain Message Parsing Tables 

[A)Maintain Geographic Reference and Message 

Datum Tables] 

Y Y 
22

Table 5. Training evaluation debrief data collection (continued).

Task No. Task Name 

Instructional 

Difficulties 

Learning 

Difficulties 

RWS-0613 Maintain User Defined Database 

[A)Create a New Table, B)Create/Delete/Modify 

Attributes for a Table, C)Select Existing Value Set, 

D)Create New Value Set, E)Publish User Defined 

Table, F)Delete User Defined Tables(Published/Not 

Published), G)Modify Privileges in a Table, H)Access 

Table Maintenance for User Defined Databases, 

I)Input Data to User Defined Table] 

Y Y 

RWS-0614 Use Table Maintenance Miscellaneous Functions Y Y 

RWS-0615 Establish JCDB Criteria 

[A)Access JCDB Criteria, B)Deactivate Auto 

Forward of ASCDB Data to JCDB, C)Establish 

Criteria for Forwarding Data to JCDB] 

Y Y 

RWS-0616 Perform External Database Coordination (EDC) 

Synchronization 

[A)Request an EDC, B)Auto Forward an EDC, 

C)Manually Define Criteria, D)Send an EDC] 

Y Y 

RWS-0617 Archive, Purge, and Restore Data Y Y 

RWS-0700 Process Target Criteria(D281) Messages 

[A)Review/Delete Target Criteria, B)Associate 

TIDAT from Alerts, C)Plot Target Criteria Area from 

Alerts, D)Create Alert Criteria] 

Y Y 

RWS-0701 Build and Manage Target Sheets 

[A)Create/Delete/Edit Target Sheet, B)Create Alert 

Criteria] 

Y Y 

RWS-0702 Nominate Targets 

[A)Nominate Target from Active Target List 

Processing Queue/Database Operations] 

Y Y 

RWS-0703 Maintain Target Parameters 

[A)Set Target Parameters, B)Establish/Maintain 

Target Number Block] 

Y Y 

RWS-0704 Process Target Coordination Messages 

[A)Process Target Coordination Request from 

Alert/Directory, B)Register Target Coordination 

Request Alerts in Multimode] 

Y Y 

RWS-0705 Manage Targeting Queues 

[A)Nominate Target from Targeting Queue, 

B)Save/Modify/Delete Nominated Entity] 

Y Y 

RWS-0706 Maintain Target Type(Equipment/ 

Facility) Translation Tables 

Y Y 
23

ACE TRAINING QUESTIONNAIRE

Purpose: This questionnaire is used to capture your views concerning your ACE Functional Identity (FI)

Classroom Training. The data you provide will help decision makers evaluate how effective the Training program

has been designed to effectively train soldiers to use the ACE. None of the data you give will be provided to your

unit or higher level chain of command nor entered into your personal files. Your responses will be maintained under

strict enforcement of Confidentiality Public Law.

Instructions: Read each item carefully. Enter your response in the space provided or circle the appropriate

response. We would welcome any written comment you may provide in response to a particular question asked.

Please use the reverse of the page (if necessary) for these entries and indicate item number for which you are giving

written comment.

Rating Scheme: The rating criteria used in this questionnaire are defined below. Please ensure that you understand

these rating criteria when answering questions with a rating scheme. Circle number which indicates your

judgment--or N/A = 9 if item is not relevant for you.

1: Completely Agree: There must be absolutely no doubt when using this response that the factor being

evaluated cannot be any better--in condition, design or composition and is a desirable system feature.

2: Strongly Agree: This response indicates that the factor being evaluated is very good and very helpful to the

analyst/operator.

3: Generally Agree: This response indicates the factor being evaluated is acceptable and helpful to the

analyst/operator.

4: Generally Disagree: This response indicates the factor being evaluated is unacceptable, but only minor

improvements are required to make it acceptable. Written comment indicating examples of how change should

be made would be appropriate when this rating is used.

5: Strongly Disagree: This response indicates that the factor being evaluated is unacceptable and major

improvements are required to make it acceptable. Written comment indicating examples of how change should

be made would be appropriate when this rating is used.

6: Completely Disagree: There must be absolutely no doubt when using this response that the factor being

evaluated is unacceptable--condition, design or composition and must be completely changed to be acceptable.

Written comment indicating examples of how change should be made would be appropriate when this rating is

used.
24

PART I. GENERAL INFORMATION

1. Dates Training Received: (MM/DD/YY) _________________________________________

2. Course Name/FI Training: ____________________________

3. Student Name/Rank: _________________________________________________________

PART II. COURSE MATERIAL

4. Information received during this training will:

1 = Enhance my ability to do my job

2 = Somewhat enhance my ability to do my job

3 = Be of no assistance in doing my job

5. Technical information presented during this training was:

1 = Incomprehensible

2 = Elementary, boring

3 = Comprehensible, easily understood

6. This training was:

1 = Too long

2 = Too short

3 = About right

7. Reference question 6: If any part of the classroom instruction was too long, what part(s) could be shortened and

still meet your training needs?

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

8. Reference question 6: If any part of the classroom instruction was too short, what part(s) should be lengthened

to meet your training needs?

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

9. Position manuals were available for reference for all parts of the instruction if I had difficulty executing a

function.

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=9

Indicate part(s) where manuals were not available. ____________________________________

______________________________________________________________________________

10. Appropriate field manuals were available for reference for all parts of instruction to accomplish analytical

tasks if needed.

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=9

Indicate part(s) where manuals were not available. ____________________________________

______________________________________________________________________________

11. Appropriate technical manuals were available for reference for all parts of instruction to accomplish operator

level maintenance and or troubleshooting tasks when needed and/or required.

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=9
25

Indicate part(s) where manuals were not available. ____________________________________

______________________________________________________________________________

12. All manuals are easy to understand.

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=9

Indicate those manuals which are not. ______________________________________________

______________________________________________________________________________

13. All manuals are organized so that information can be quickly referenced and the information found and used in

a timely manner.

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=9

Indicate those manuals which are not. ______________________________________________

______________________________________________________________________________

PART III. COURSE PRESENTATION

14. Majority of lessons taught were:

1 = Very well presented

2 = Adequately presented

3 = Poorly presented

Indicate those lessons which were poorly presented. ___________________________________

______________________________________________________________________________

15. Instructor presentations were:

1 = Well organized and easy to follow

2 = Unorganized; difficult to follow

3 = Some areas (or critical tasks) need improvement (please list) _____________________

______________________________________________________________________________

16. Instructor presentations were:

1 = Clear; easy to understand

2 = Unclear; difficult to understand

3 = Some areas (or critical tasks) required greater clarification (please list) _____________

______________________________________________________________________________

17. The instructor was:

1 = Well prepared

2 = Somewhat prepared [please list areas (or critical tasks) not prepared] ______________

______________________________________________________________________________

3 = Poorly prepared

4 = Not prepared
26

18. The instructor was:

1 = Able to answer questions or provide assistance as needed

2 = Partially able to answer questions or provide assistance as needed

3 = Unable to answer questions or provide assistance as needed for some part(s)

Indicate part(s) where questions could not be answered. ________________________________

______________________________________________________________________________

19. Practice (hands on training) needed to develop individual skills proficiency in each part of instruction was:

1 = Most sufficient

2 = Sufficient

3 = Partially sufficient

4 = Insufficient

20. Reference question 19: If hands-on training was insufficient to build operator proficiency, in what subject

area(s) (or critical tasks) did this occur?

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

21. The instructor used available training time:

1 = Fully and effectively

2 = Fully and effectively on the average

3 = Excessive amount of time was lost

PART IV. VALUE OF TRAINING AIDS

22. During this training, training aids were:

1 = Used effectively

2 = Used ineffectively

3 = Not used at all, but needed

4 = Not used at all, not needed

5 = Some training aids need further work (please list one or two examples) _____________

______________________________________________________________________________

23. Training aids were used:

1 = Too often

2 = About right

3 = Too little

24. Training aids used were:

1 = Of good quality

2 = Of fair quality

3 = Of poor quality

Recommended changes (if any) ____________________________________________________

______________________________________________________________________________
27

PART V. TOOLS AND EQUIPMENT

25. I experienced no software malfunctions which would cause me operational difficulties in the accomplishment

of intelligence gathering, processing, analyzing, and dissemination tasks.

Completely Disagree 1 2 3 4 5 6 Completely Agree N/A=9

Indicate software malfunctions experienced. _________________________________________

______________________________________________________________________________

26. I experienced no hardware malfunctions which would cause me operational difficulties in the accomplishment

of intelligence gathering, processing, analyzing, and dissemination tasks.

Completely Disagree 1 2 3 4 5 6 Completely Agree N/A=9

Indicate hardware malfunctions experienced. _________________________________________

______________________________________________________________________________

27. My workstation was of:

1 = Excellent design

2 = Good design

3 = Poor design

Please indicate recommendations for change. _________________________________________

______________________________________________________________________________

PART VI. TRAINING FACILITIES/FACTORS

28. Classroom was:

1 = Adequate in size

2 = Inadequate in size (please indicate why) ______________________________________

______________________________________________________________________________

29. Classroom was:

1 = Free of noise and other distractions

2 = Contained noise and other distractions (please indicate sources) ___________________

______________________________________________________________________________

30. Classroom was:

1 = Adequately lighted

2 = Not adequately lighted

31. Classroom was:

1 = Adequately ventilated

2 = Inadequately ventilated
28

32. Unit requirements caused disruptions to my training:

1 = Often

2 = Frequently

3 = Sometimes

4 = Never

List types of disruptions. _________________________________________________________

______________________________________________________________________________

33. Instructor to student ratios, particularly during practical exercises:

1 = Contributed to major learning difficulties

2 = Caused no learning difficulty

3 = Caused minor learning difficulty

Indicate why (if you know) there were learning difficulties. _____________________________

______________________________________________________________________________

34. Workstation shortages, particularly during hands-on training:

1 = Contributed to an inability to build individual skills proficiency

2 = Caused minor problems in building individual skills proficiency

3 = Did not interfere with building individual skills proficiency

4 = There was no workstation shortage

PART VII. PRACTICAL EXERCISES/PERFORMANCE EVALUATIONS

35. The majority of the practical exercises (PEs) in this training:

1 = Greatly assisted in building operator proficiency

2 = Moderately assisted in building operator proficiency

3 = Did not contribute to building operator proficiency

Indicate those which did not. _______________________________________________

4 = There was no practical exercise(s) during this block of instruction

36. The majority of the practical exercises in this block on instruction:

1 = Made sense. I always knew what I was doing and why. I could always relate the PE to

the accomplishment of individual tasks at my unit.

2 = Made little sense. I never knew what I was doing or why. Could not always relate the

PE to the accomplishment of individual tasks at my unit.

3 = There was no practical exercise(s) during this block of instruction.

Indicate those which did not. ______________________________________________

4 = Needed some explanation by instructor.

37. Performance tests evaluated (Check all which apply):

1 = What was taught and nothing more

2 = More than what was taught

3 = My knowledge of the system at this point of training

4 = My level of proficiency at this point of training

5 = There was no performance test(s) administered during this block of instruction
29

PART VIII. PERFORMANCE OF CRITICAL TASKS--Use TASK No. on “Training Assessment Debrief”

38. List those critical tasks taught during this training which you found difficult to learn and hard to perform:

Difficult to Learn Hard to Perform

_____________________________________ ____________________________________

_____________________________________ ____________________________________

_____________________________________ ____________________________________

_____________________________________ ____________________________________

39. I believe the following critical task(s) requires less instruction time to train.

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

40. I believe the following critical task(s) requires more instruction time to train.

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

41. I believe the following critical task(s) will require continuous practice and training at unit level to prevent skill

decay and/or proficiency loss.

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

PART IX. OPINION, REMARKS, RECOMMENDATIONS

42. Training received during this training was:

1 = Very good

2 = Good

3 = Fair

4 = Poor

43. I recommend the following additions to this training. Why?

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

44. I recommend the following deletions to this training. Why?

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________
30

Analysis and Control Element (ACE)

Initial Operational Test and Evaluation (IOTE)

Demographic Questionnaire

Purpose: This questionnaire is used to capture information related to your military background, MOS experience,

and your operational familiarity with current intelligence gathering systems.

PRIVACY ACT STATEMENT

Public Law 93-573, called the Privacy Act of 1974, requires that you be informed of the purpose and uses to

be made of the information that is collected.

The Department of the Army may collect the information requested under authority of 10 United States Code

137.

Providing information in this questionnaire is voluntary. Failure to respond to any particular question will

not result in any penalty for the respondent.

The information collected in this survey will be used to evaluate military intelligence systems and their

utilization.

The information will be used for research and analysis only . The US Army Operational Evaluation

Command has primary research, analysis, and evaluation responsibility.

Instructions: Read each item carefully. Enter your response in the space provided.

Last Name: ____________________________________ First Name:_______________________________ MI:_____

(Please Print) (Please Print)

SSN: __________________ Approval

1________________________________________

Signature

Date: _____ _____ _____ Operator Echelon: ___________________________________

MO DA YR (Bde, Div, etc.)

Functional Identity (ACE Function)_________________________________________________

 1SSN is requested only to make it possible to obtain your ASVAB Subtest and Composite Scores from the Defense 

Manpower Data Center (DMDC) in Monterey, CA. ASVAB data and other personal data on file at DMDC will be used to 

indicate how comparable you and other soldiers participating in this test are to all soldiers who will be using ACE when it is 

fielded. 
31

I. MILITARY INFORMATION:

1. Rank: ___________

2. Primary MOS: ___________

3. Secondary MOS: ___________

4. Duty MOS/SSI: ___________

5. Date Graduated from AIT/OBC: __ __ / __ __ (MM/YR)

6. Date Received PMOS/SSI: __ __ / __ __ (MM/YR)

7. How did you receive your PMOS/SSI (Please Circle Response):

1=AIT 2=OJT 3=Reclassification 4=Promotion 5=Other ___________________________

8. Date Entered Active Duty: __ __ / __ __ (MM/YR)

9. Unit Assigned: _________________________

10. Date Assigned: __ __ / __ __ (MM/YR)

11. ETS/ESA: __ __ / __ __ (MM/YR)

12. Projected PCS Date: __ __ / __ __ (MM/YR)

13. Current Duty Position: ___________________________ (outside the WFX)

14. Time in Current Duty Position: __ __ (MM)

15. Time Using Army C2 Digital Systems: __ __ (MM)

16. Civilian Education (Please Circle Response):

a. Current b. At Last Service Entry

1= No high school degree 1= No high school degree

2= High school diploma 2= High school diploma

3= GED 3= GED

4= Some college (1-2 years) 4= Some college (1-2 years)

5= Technical school 5= Technical school

6= Associate Degree 6= Associate Degree

7= Three or more years of college (no degree) 7= Three or more years of college (no

degree)

8= College degree 8= College degree

9= Graduate degree 9= Graduate degree

17. Gender 1= Male 2= Female 18. Age ______ yrs ______months

19. Height _______ inches 20. Weight _______ lbs

21. Eyeware (during evaluation session): None ___ Glasses ___ Contacts __

22. Handedness: Right ___ Left ___ Ambidextrous ___
32

23. Racial/Ethnic Background 

1= American Indian/Alaskan Native

2= Asian/Pacific Islander

3= Black, not Hispanic origin

4= Hispanic

5= White, not Hispanic origin

24. Current Injuries (list) _____________________________________________________________________

II. INTELLIGENCE SYSTEMS EXPERIENCE: The following questions relate to your automated intelligence processing

systems experience. If you have received training on any of the systems cited please identify the nature of that experience in the

area specified.

1. Type System: EPDS 1 = Yes 2 = No

Type Training: Year Trained:__________

1 = AIT 2 = NETT 3 = OJT 4 = Other:_____________ (Specify)

Experience on System Years/Months (YR/MM) __ __ / __ __

2. Type System: TCAC 1 = Yes 2 = No

Type Training: Year Trained:__________

1 = AIT 2 = NETT 3 = OJT 4 = Other:_____________ (Specify)

Experience on System Years/Months (YR/MM) __ __ / __ __

3. Type System: THMT 1 = Yes 2 = No

Type Training: Year Trained:__________

1 = AIT 2 = NETT 3 = OJT 4 = Other:_____________ (Specify)

Experience on System Years/Months (YR/MM) __ __ / __ __

4. Type System: MICROFIX 1 = Yes 2 = No

Type Training: Year Trained:__________

1 = AIT 2 = NETT 3 = OJT 4 = Other:_____________ (Specify)

Experience on System Years/Months (YR/MM) __ __ / __ __

5. Type System: GUARDRAIL 1 = Yes 2 = No

Type Training: Year Trained:__________

1 = AIT 2 = NETT 3 = OJT 4 = Other:_____________ (Specify)

Experience on System Years/Months (YR/MM) __ __ / __ __

6. Type System: JSTARS 1 = Yes 2 = No

Type Training: Year Trained:__________

1 = AIT 2 = NETT 3 = OJT 4 = Other:_____________ (Specify)

Experience on System Years/Months (YR/MM) __ __ / __ __
33

7. Type System: QUICKFIX 1 = Yes 2 = No

Type Training: Year Trained:__________

1 = AIT 2 = NETT 3 = OJT 4 = Other:_____________ (Specify)

Experience on System Years/Months (YR/MM) __ __ / __ __

8. Type System: TEAMMATE 1 = Yes 2 = No

Type Training: Year Trained:__________

1 = AIT 2 = NETT 3 = OJT 4 = Other:_____________ (Specify)

Experience on System Years/Months (YR/MM) __ __ / __ __

9. Type System: TACJAM 1 = Yes 2 = No

Type Training: Year Trained:__________

1 = AIT 2 = NETT 3 = OJT 4 = Other:_____________ (Specify)

Experience on System Years/Months (YR/MM) __ __ / __ __

10. Type System: UAV 1 = Yes 2 = No

Type Training: Year Trained:__________

1 = AIT 2 = NETT 3 = OJT 4 = Other:_____________ (Specify)

Experience on System Years/Months (YR/MM) __ __ / __ __

11. Type System: TRAILBLAZER 1 = Yes 2 = No

Type Training: Year Trained:__________

1 = AIT 2 = NETT 3 = OJT 4 = Other:_____________ (Specify)

Experience on System Years/Months (YR/MM) __ __ / __ __

12. Type System: ASAS 1 = Yes 2 = No

Type Training: Year Trained:__________

1 = AIT 2 = NETT 3 = OJT 4 = Other:_____________ (Specify)

Trained on ASAS Subsystems: 1) SS 2) AS 3) RWS 4) ASAS Light 5) CCS

6) ACT-E 7) CI/HUMINT 8) Trusted Work Stations

9) MASINT 10) OSINT

Experience on System Years/Months (YR/MM) __ __ / __ __
34

III. COMPUTER EXPERIENCE:

13. How many months experience have you had with a Personal Computer/MAC or Laptop/Notebook computer? ___________

14. Confidence in: Very Somewhat Not

Confident Confident Confident Confident

Using Computers in General ___________ ___________ ___________ ___________

Using Personally Owned Computers ____________ ___________ ____________ ___________

Using Army C2 Digital Systems ____________ ___________ ___________ ___________

Performing Multiple Tasks at Same Time ____________ ___________ ____________ ___________

15. Average Number of Hours Per Week Spent Using a Computer (On/Off Duty) ___________(hours)

IV. EXPERIENCE WITH SPECIFIC INTELLIGENCE FUNCTIONS:

16. Estimate your knowledge in the use of JINTACCS/USMTF message formats.

0 = 0% 1 = 25% 2 = 50% 3 = 75% 4 = 100%

17. Please record the number of years/months you have actually performed the following function at the highest command level.

If sections under the topical headings are not applicable to you, circle “2=No” adjacent to the sub-topic area.

COLLECTION MANAGEMENT: If Function is Yes, indicate: Experience

YY MM

Requirements Management 1=Yes 2=No Command Level: 1=Bn 2=Bde 3=Div 4=Corp 5=EAC ____ ____

Requirements Development 1=Yes 2=No Command Level: 1=Bn 2=Bde 3=Div 4=Corp 5=EAC ____ ____

Collection Evaluation

Analysis 1=Yes 2=No Command Level: 1=Bn 2=Bde 3=Div 4=Corp 5=EAC ____ ____

MISSION MANAGEMENT:

Resource Selection 1=Yes 2=No Command Level: 1=Bn 2=Bde 3=Div 4=Corp 5=EAC ____ ____

Non-Organic Support

Requests 1=Yes 2=No Command Level: 1=Bn 2=Bde 3=Div 4=Corp 5=EAC ____ ____

ASSET MANAGEMENT:

Organic Resource Task 1=Yes 2=No Command Level: 1=Bn 2=Bde 3=Div 4=Corp 5=EAC ____ ____ Resource

Status Monitoring 1=Yes 2=No Command Level: 1=Bn 2=Bde 3=Div 4=Corp 5=EAC ____ ____

INTELLIGENCE MESSAGE PROCESSING:

Message Processing-IN 1=Yes 2=No Command Level: 1=Bn 2=Bde 3=Div 4=Corp 5=EAC ____ ____

Message Processing-OUT 1=Yes 2=No Command Level: 1=Bn 2=Bde 3=Div 4=Corp 5=EAC ____ ____

Message Preparation-OUT 1=Yes 2=No Command Level: 1=Bn 2=Bde 3=Div 4=Corp 5=EAC ____ ____

Processing 1=Yes 2=No Command Level: 1=Bn 2=Bde 3=Div 4=Corp 5=EAC ____ ____

Battle Damage Assessment

Input 1=Yes 2=No Command Level: 1=Bn 2=Bde 3=Div 4=Corp 5=EAC ____ ____

All Source Database 1=Yes 2=No Command Level: 1=Bn 2=Bde 3=Div 4=Corp 5=EAC ____ ____

Order of Battle 1=Yes 2=No Command Level: 1=Bn 2=Bde 3=Div 4=Corp 5=EAC ____ ____

ALL SOURCE PROCESSING:

MTI Processing 1=Yes 2=No Command Level: 1=Bn 2=Bde 3=Div 4=Corp 5=EAC ____ ____

FTI Processing 1=Yes 2=No Command Level: 1=Bn 2=Bde 3=Div 4=Corp 5=EAC ____ ____

HUMINT Processing 1=Yes 2=No Command Level: 1=Bn 2=Bde 3=Div 4=Corp 5=EAC ____ ____

New Nodes Processing 1=Yes 2=No Command Level: 1=Bn 2=Bde 3=Div 4=Corp 5=EAC ____ ____

Node Change Status

SITUATION DEVELOPMENT:

Dynamic Situation Monitoring 1=Yes 2=No Command Level: 1=Bn 2=Bde 3=Div 4=Corp 5=EAC ____ ____

IPB 1=Yes 2=No Command Level: 1=Bn 2=Bde 3=Div 4=Corp 5=EAC ____ ____
35

TARGET DEVELOPMENT:

Target Nominations 1=Yes 2=No Command Level: 1=Bn 2=Bde 3=Div 4=Corp 5=EAC ____ ____

Battle Damage Assessment 1=Yes 2=No Command Level: 1=Bn 2=Bde 3=Div 4=Corp 5=EAC ____ ____

ELECTRONIC WARFARE SUPPORT:

EW Planning 1=Yes 2=No Command Level: 1=Bn 2=Bde 3=Div 4=Corp 5=EAC ____ ____

Evaluation of Current

Operations 1=Yes 2=No Command Level: 1=Bn 2=Bde 3=Div 4=Corp 5=EAC ____ ____

Evaluation of MIJI Reports 1=Yes 2=No Command Level: 1=Bn 2=Bde 3=Div 4=Corp 5=EAC ____ ____

OPSEC SUPPORT:

Risk Assessment 1=Yes 2=No Command Level: 1=Bn 2=Bde 3=Div 4=Corp 5=EAC ____ ____

Counter Actions Planning 1=Yes 2=No Command Level: 1=Bn 2=Bde 3=Div 4=Corp 5=EAC ____ ____

REPORT EVALUATING

Effectiveness 1=Yes 2=No Command Level: 1=Bn 2=Bde 3=Div 4=Corp 5=EAC ____ ____

SIGINT ANALYSIS:

COMINT/Traffic Analysis 1=Yes 2=No Command Level: 1=Bn 2=Bde 3=Div 4=Corp 5=EAC ____ ____

CRYPTO Analysis 1=Yes 2=No Command Level: 1=Bn 2=Bde 3=Div 4=Corp 5=EAC ____ ____

Signals Analysis 1=Yes 2=No Command Level: 1=Bn 2=Bde 3=Div 4=Corp 5=EAC ____ ____

COMINT/ELINT Integration 1=Yes 2=No Command Level: 1=Bn 2=Bde 3=Div 4=Corp 5=EAC ____ ____

ELINT/Signals Analysis 1=Yes 2=No Command Level: 1=Bn 2=Bde 3=Div 4=Corp 5=EAC ____ ____

ASSET MANAGEMENT

Organic Resource Tasking 1=Yes 2=No Command Level: 1=Bn 2=Bde 3=Div 4=Corp 5=EAC ____ ____

Resource Status Monitoring 1=Yes 2=No Command Level: 1=Bn 2=Bde 3=Div 4=Corp 5=EAC ____ ____
36

2.3.1.2.4 MANPRINT Debrief (table 6). The form—referenced in row 4, column 1 of table 4—

presented as an example represents an iteration of the form used for collective tasks in a later 

LUTE for a different ASAS subsystem. Generally, MANPRINT Debrief forms are used for 

Individual—keystroke-oriented critical tasks, as well as Collective critical tasks. Collective 

tasks generally involve sequences of Individual tasks and are more easily identified as Missionoriented MOS-based tasks. This particular form (see page 37) provides more detail about 

problems that may have existed in the training process that could be attributed to the disparity in 

the way tasks are performed with and without computer augmentation. Originally, it was 

planned to be administered at the end of each test day in the Pilot and Record test for analysts 

and data collectors. The data collectors received the same training as the analysts and they stood 

by the analysts during the test and watched what they were doing. The MANPRINT Debrief was 

for them to document the individual critical tasks that were performed and to indicate those tasks 

that were problematic. While this form was designed to be used stand-alone with appropriate 

comments written on the form, it was used more as a tabulation sheet for soldiers’ referral so that 

they knew those individual and collective tasks on which a more complete MANPRINT 

Evaluation was required—on the next form discussed. 

2.3.1.2.5 MANPRINT Evaluation. The form—referenced in row 5, column 1 on the first page of 

table 4—is used in conjunction with or after the MANPRINT Debrief. The MANPRINT 

Debriefs are forms that contains key items that can apply to each of the critical tasks and asks the 

respondents to provide detailed feedback about problems in their performance (or what they 

observed). They use the completed form (shown in table 6) as a reference for determining the 

number of MANPRINT evaluation forms they are expected to complete (see page 44). Those 

forms were used to provide detailed ratings and comments on problematic tasks. 

In appendix G, a standalone Workload Questionnaire used during the MCS IOTE is presented 

(18). 

2.3.1.2.6 Human Factors Engineering General Software Functioning. The particular version of 

this form presented here—referenced in row 1, column 1 of the second page of table 4—was 

modified for use in the block 2 IOTE testing for ASAS (see page 48). It was adapted from a 

similar instrument used in earlier ASAS sub-system LUTE testing. Items included on this form 

addressed the adequacy of different characteristics of the: (1) video display; (2) keyboard; and 

(3) mechanics of accessing software. This form was generally completed near the end of the test 

so that soldiers would have had maximal opportunity to make judgments about the video, 

keyboard and accessing software. 
37

MANPRINT DEBRIEF FOR 

ASAS-LIGHT BLOCK 2 LUTE 

(COLLECTIVE TASKS) 

BIO/BACKGROUND

Name_________________________ PIN no.___________ DATE ______________ 

 Please Print Mo/Da/Yr 

 Position: ASAS Light Analyst 

 Echelon (please circle): 

 BTN DIV BDE CORP EAC 

INSTRUCTIONS FOR COLLECTIVE CRITICAL TASKS

1. Each listed TASK has sub-tasks, many of which are completed when the TASK is performed 

(columns A and B). 

2. For each TASK (and sub-TASK), circle Y in the PERFORMED column (column C), if performed during 

collective training. 

3. Circle Y in the PROBLEMS column (column D) if you experienced one or more problems in performing this 

TASK (or sub-TASK). 

4. In column E (1), circle Y if your training indicated which individual tasks you needed to use to perform the 

TASK (or sub-TASK). 

5. In column E (2), indicate the ADEQUACY of training you received in HOW TO USE those INDIVIDUAL 

TASKS to complete each Collective Task (and sub-Task) using the following scale: 

 1 = More than adequate 

 2 = Adequate 

 3 = Not quite adequate 

 4 = Barely adequate 

 5 = Not adequate
38

Table 6. MANPRINT debrief for collective tasks. 

A 

Task 

No. 

B 

Task Name 

(Collective) 

C 

Performed 

D 

Problems 

E 

Training Identified Individual 

Tasks Needed to Perform

Which 

Tasks 

(1)

How to Use Tasks

(Adequacy) 

(2)

2003 Conduct Intelligence 

Preparation of the Battlefield 

(IPB) (MNVR BN) 

Y Y Y Y 

-A Establish coordination with 

other staff elements IAW 

unit tactical SOP 

Y Y Y Y 

-B Define the battlefield area Y Y Y Y 

-C Analyze/describe 

terrain/weather effects on 

friendly and enemy COAs 

Y Y Y Y 

-D Prepare a modified 

combined obstacle overlay 

Y Y Y Y 

-E Create/update threat 

doctrine or patterns of 

operation to doctrinal 

templates 

Y Y Y Y 

-F Identify threats likely 

objectives/end state one and 

two levels of threat 

command below your own 

Y Y Y Y 

-G Assist in production of 

decision support template 

through wargaming and 

other developed IPB 

products 

Y Y Y Y 

-H Confirm/deny/update 

existing estimate of enemy's 

COA 

Y Y Y Y 

2004 Produce Intelligence Products 

(MNVR BN) 

Y Y Y Y 

-A Record incoming 

information and intelligence 

Y Y Y Y 

-B Provide timely intelligence 

support to targeting 

Y Y Y Y 

-C Dynamically establish/ 

terminate sensor-to-shooter 

links 

Y Y Y Y 

-D Produce the intelligence 

annex to the battalion 

OPORD 

Y Y Y Y 

-E Process/disseminate the 

Intelligence Annex 

Y Y Y Y 

-F Produce an intelligence 

summary addressing 

commander's PIR and IR 

Y Y Y Y 
39

Table 6. MANPRINT debrief for collective tasks (continued). 

A 

Task 

No. 

B 

Task Name 

(Collective) 

C 

Performed 

D 

Problems 

E 

Training Identified Individual 

Tasks Needed to Perform

Which 

Tasks 

(1)

How to Use Tasks

(Adequacy) 

(2)

2005 Disseminate Combat 

Information and Intelligence 

(MNVR BN) 

Y Y Y Y 

-A Determine 

combat/intelligence 

information that requires 

dissemination

Y Y Y Y 

-B Disseminate combat 

information to addresses 

listed in tactical SOP 

Y Y Y Y 

-C Disseminate via frequency 

modulated /MSE, tactical 

satellite, tactical internet... 

Y Y Y Y 

-D Confirm receipt of 

information/intelligence 

passed

Y Y Y Y 

2009 Intelligence Operations 

(MNVR BN) 

Y YY Y

-A Monitor current/projected 

enemy situation and COA 

Y YY Y

-B Recommend changes to PIR Y Y Y Y

-C Recommend changes to IR Y Y Y Y

-D Supervise intelligence 

acquisition tasks by 

battalion assets 

Y Y Y Y 

-E Monitor status of 

intelligence information 

requests 

Y Y Y Y 

-F Initiate new intelligence 

information requests 

Y YY Y

-G Supervise transfer of 

captured personnel/ 

documents/ material IAW 

brigades tactical SOP 

Y Y Y Y 

-H Coordinate intelligence 

effort between main and 

tactical command post

Y Y Y Y 

-I Supervise release/ 

dissemination of intelligence 

products 

Y Y Y Y 

2010 Maintain the Current Enemy 

Situation (MNVR BN) 

Y YY Y

-A Merge significant aspects of 

AO and current enemy 

situation 

Y Y Y Y 

-B Use PIR/IR in analysis of 

current enemy situation 

Y YY Y
40

Table 6. MANPRINT debrief for collective tasks (continued). 

A 

Task 

No. 

B 

Task Name 

(Collective) 

C 

Performed 

D 

Problems 

E 

Training Identified Individual 

Tasks Needed to Perform

Which 

Tasks 

(1)

How to Use Tasks

(Adequacy) 

(2)

-C Analyze/compare current 

enemy dispositions/ 

compositions with project 

action course 

Y Y Y Y 

-D Confirm/deny courses of 

action/update enemy 

situation and track status of 

LTIOV for each PIR 

Y Y Y Y 

-E Maintain current enemy 

situation (capabilities 

/vulnerabilities) with 

prioritized COA and 

probable future intent

Y Y Y Y 

-F Maintain situation map to 

show enemy situation and 

locations and indicators of 

future events 

Y Y Y Y 

2012 Supervise Intelligence 

Operations (MNVR BN)

Y YY Y

-A Serve as focus for battalions 

intelligence support

Y YY Y

-B Conduct Briefings of 

assigned elements on 

current situation and new 

taskings 

Y YY Y

-C Review taskings for clarity 

and completeness 

Y YY Y

-D Prioritize intelligence 

requirements 

Y YY Y

-E Supervise efforts in support 

of IPB process 

Y YY Y

-F Submit intelligence products 

ready for dissemination 

Y YY Y

2013 Process Specific Information 

Requirements (SIR) Data 

(MNVR BN)

Y Y Y Y 

-A Identify enemy 

characteristics that meet 

specific information 

requirements 

Y Y Y Y 

-B Integrate specific 

information requirements 

into reconnaissance, 

intelligence, surveillance 

and target acquisition plan 

Y Y Y Y 

-C Identify information/ 

intelligence gaps 

Y YY Y

-D When will enemy 

reconnaissance elements 

move 

Y Y Y Y 
41

Table 6. MANPRINT debrief for collective tasks (continued).

A 

Task 

No. 

B 

Task Name 

(Collective) Performed Problems 

(E) 

Training Identified Individual 

Tasks Needed to Perform

Which 

Tasks 

(1)

How to Use Tasks

(Adequacy) 

(2)

-E Indicate specifics on 

enemy artillery batteries 

Y Y Y Y 

-F Forward SIR/PIR/JR to 

brigade S2 

Y YY Y

-G Determine intelligence 

reliability source/agency

Y YY Y

-H Determine intelligence 

credibility

Y YY Y

-I Compare incoming data 

with intelligence estimate 

Y YY Y

-J Compare incoming data 

with intelligence products 

developed during IPB 

process (intel indicators, 

intel workbook, PIRs, IRs, 

SIRs, Situation map)

Y Y Y Y 

-K Determining validity of 

incoming data based on 

preceding standards 

Y Y Y Y 

-L Analyze enemy 

information/intelligence 

(re: 

strengths/vulnerabilities, 

weather terrain...)

Y Y Y Y 

-M Provide estimate of 

enemy's ability/likeliness 

to use NBC 

Y Y Y Y 

-N Acquire information to 

conduct/develop command 

information, operations 

security, and electronic 

warfare analysis and a 

tactical deception plan 

Y Y Y Y 

-O Estimate potential 

effectiveness of enemy 

smoke and NBC weapons 

on friendly operations 

Y Y Y Y 

2015 Provide Intelligence Support 

to Targeting (MNVR BN)

Y Y Y Y 

“Decide” Function of 

Targeting 

-A Develop high value/payoff 

target lists 

Y Y Y Y 

-B Develop target selection 

standards and management 

matrix 

Y Y Y Y 

-C Determine whether 

available assets and time 

constraints will make a 

target attackable 

Y Y Y Y 
42

Table 6. MANPRINT debrief for collective tasks (continued).

A 

Task 

No. 

B 

Task Name 

(Collective) Performed Problems 

(E) 

Training Identified Individual 

Tasks Needed to Perform

Which 

Tasks 

(1)

How to Use Tasks

(Adequacy) 

(2)

-D Develop attack guidance 

matrix from decision 

support template and time 

phase lines 

Y Y Y Y 

“Detect”Function of 

Targeting 

 

-E Provide FSO all targetable 

data 

Y Y Y Y 

-F Refine reconnaissance, 

intelligence, surveillance 

and target acquisition plan 

Y Y Y Y 

-G Inform FSO and brigade 

staff of tactical changes by 

enemy 

Y Y Y Y 

-H Notify FSO of changes in 

common relevant picture 

or situation map that effect 

targeting plan 

Y Y Y Y 

-I Update decision support 

template to support attack 

guidance matrix 

Y Y Y Y 

 “Deliver” Function of 

Targeting 

 

-J Provide targeting team 

with location and targeting 

data 

Y Y Y Y 

 “Assess” Function of 

Targeting 

 

-K Determine which targets 

require BDA 

Y Y Y Y 

-L Task assets to collect BDA Y Y Y Y 

-M Analyze and horizontally 

disseminate BDA results 

per battalion SOP 

Y Y Y Y 

-N Recommend battle plan 

change based on BDA 

Y Y Y Y 

2020 Produce a Reconnaissance, 

Intelligence, Surveillance 

and Target Acquisition 

(RISTA) Plan (MNVR BN) 

Y Y Y Y 

-A Associate RISTA 

objectives, PIRs, situation 

and event templates to 

indicate expected enemy 

COA 

Y Y Y Y 
43

Table 6. MANPRINT debrief for collective tasks (continued). 

A 

Task 

No. 

B 

Task Name 

(Collective) Performed Problems 

(E) 

Training Identified Individual 

Tasks Needed to Perform

Which 

Tasks 

(1)

How to Use Tasks

(Adequacy) 

(2)

-B Identify available 

collection assets 

Y Y Y Y 

-C Prepare intelligence 

synchronization matrix to 

answer PIRs, targeting 

requirements and RISTA 

objectives 

Y Y Y Y 

-E Ensure mix of assets is 

used 

Y Y Y Y 

-F 

Create RISTA overlay 

with RISTA matrix to 

address “who”, “what”, 

“where”, “when” and 

“how” 

Y Y Y Y 

-G Use RISTA plan to 

develop SIRs for each PIR 

Y Y Y Y 

2032 Conduct Intelligence 

Functions for Deployment 

Y Y Y Y 

-A Analyze AO Y Y Y Y 

-B Collect intelligence 

products (imagery, terrain, 

weather) 

Y Y Y Y 

-C Conduct IPB Y Y Y Y 

-D Identify intelligence 

database gaps 

Y Y Y Y 

-E Prepare intelligence 

estimate/develop threat 

models 

Y Y Y Y 

-F Recommend PIRs and IRs 

to support operation 

concept 

Y Y Y Y 

1

Each task no. is actually preceded by “34-1-”and followed by “34-00MA”. 
44

MANPRINT EVALUATION 

ANALYSIS AND CONTROL ELEMENT (ACE) 

INITIAL OPERATIONAL TEST AND EVALUATION (IOTE) 

 

BIO/BACKGROUND/POSITION

Name PIN no. _________________ Date ________________ 

 (Last 4 SSN) (Mo/Da/Yr) 

Position(circle or indicate): CCS SMART Multifunction Work Station RIS/ELINT SS AS 

 SIT/IPB DBM SIGINT Trusted Suite ISR CI/HUMINT OSINT 

 

 

Test Phase: End of Test 

Problem Description 

1. Task Performing (use task no.): 

____________________________________________________________________________________ 

2. Most probable problem causes (circle up to three most important): 

 a. Equipment malfunction (appears unrelated to any soldier action) 

 b. Training 

 (1) Poor or inadequate classroom training on task (2) Lack of sustainment/collective training on task 

 c. Manpower (not enough soldiers to do job involving this task) 

 (1) Need more analysts to perform this task (2) Need more time to complete the task 

 (3) Need more responsive maintenance support (4) Need more/better supervision 

 d. Personnel (task is outside “normal” duties for my MOS or skill level) 

 e. Health Hazard (task jeopardized my or another crew member's safety) 

 f. System Safety (task performed created a safety problem related to ASAS equipment) 

g. Human Factors 

 (1) Problem working with ACE Functional Identity equipment/hardware--difficult or complicated to 

work with or access when this task was performed 

 (2) Problem with ACE Functional Identity software when this task was performed 

 (3) Problem with ACE Functional Identity procedures
45

3. Problem Description:________________________________________________________________ 

____________________________________________________________________________________ 

____________________________________________________________________________________ 

____________________________________________________________________________________ 

____________________________________________________________________________________ 

____________________________________________________________________________________ 

4. Problem reported in (1) Individual Training (Survey) or (2) Collective Training (Survey/Interview) 

 5. Frequency this task was performed since formal class (Individual Training) was completed _________ (use 

number from following scale) 

1 = at least once daily 3 = at least once in the three weeks 

2 = once or twice a week 4 = not at all 

6. TM (D.O.G.) error contributed to this problem: YES NO (circle) 

 If YES, indicate: (a) TM no./D.O.G. (b) page/para or Figure (c) error description 

 a) TM/D.O.G.______________________________ 

 b) _________________________________ 

 c) _____________________________________________________________________________ 

____________________________________________________________________________________ 

7. Problem Frequency/Severity (when task is being performed): 

Frequency (circle letter indicating judgment) 

a. Frequent - continuously experienced 

b. Probable - will occur frequently 

c. Occasional - will occur several times 

d. Remote - unlikely, but possible 

e. Improbable - very unlikely to occur 

Severity (circle Roman numeral indicating 

judgment) 

I Catastrophic - death or system's loss 

II Critical - severe injury or major system 

 damage 

III Marginal - minor injury or system damage 

IV Negligible - less than minor injury or 

 system damage 
46

8. Job Stress (when task is being performed): 

a) TEMPORAL STRESS - Pace at which task must 

be performed is hurried or rushed 

Circle No.

1 = Not hurried or rushed 

2 = A little hurried or rushed 

3 = Occasionally hurried or rushed 

4 = Frequently hurried or rushed 

5 = Very frequently hurried or rushed 

b) MENTAL STRESS - Task completion requires 

conscious mental effort (concentration) 

Circle No.

1 = None needed 

2 = Little needed 

3 = Moderate amount needed 

4 = Extensive amount needed 

If 3 or 4, indicate why: 

_______________________________________ 

_______________________________________ 

c) PHYSICAL STRESS - Task completion requires 

physical effort. 

Circle No.

1 = No demand 

2 = Little demand 

3 = Moderate demand 

4 = Heavy demand 

5 = Very heavy demand 

If 4 or 5, indicate why: 

_______________________________________ 

_______________________________________ 

d) PSYCHOLOGICAL STRESS - Task completion 

causes confusion, frustration, or anxiety. 

Circle No.

1 = None 

2 = Little 

3 = Moderate 

4 = High 

5 = Intense 

If 4 or 5, indicate why: 

_______________________________________□____

___________________________________ 

8. Job Stress (when task is being performed): (continued) 
47

e) TASK OVERLAP - Task must be performed at or 

nearly at same time as other tasks or with interruptions. 

Circle No.

1 = No overlap (or interruptions) 

2 = Little overlap (or interruptions) 

3 = Occasional overlap (or interruptions) 

4 = Frequent overlap (or interruptions) 

5 = Very frequent overlap (or interruptions) 

If 4 or 5, which other tasks (use no. from Critical Task 

List) 

Task List ________ ________ ________ 

Task No. ________ ________ ________ 

f) EFFORT STRESS - Task completion to meet 

mission needs requires continuous effort. 

Circle No.

1 = No 

2 = Little 

3 = Moderate 

4 = Extensive 

If 3 or 4, indicate why: _____________________ 

_______________________________________ 

_______________________________________ 

_______________________________________ 

_______________________________________ 

 

9. Mission successfully performed (even with problem): 100% 75% 50% 25% 0% 

10. Fix (procedure/task modification needed to complete mission): YES NO (circle) 

11. Recommended fix (if any) (continue on back side, if necessary): ____________________________ 

____________________________________________________________________________________ 

____________________________________________________________________________________ 

____________________________________________________________________________________ 
48

HUMAN FACTORS ENGINEERING (GENERAL PC-SOFTWARE FUNCTIONING) QUESTIONNAIRE

FOR THE ANALYSIS AND CONTROL ELEMENT (ACE)

INITIAL OPERATIONAL TEST AND EVALUATION (IOTE)

Purpose: This questionnaire is used to capture your views concerning the Human Factors Engineering aspects of

ACE. The data you provide will help decision makers evaluate how effective the ACE has been designed to assist

you in the performance of your intelligence gathering, processing and analysis tasks.

PRIVACY ACT STATEMENT

Public Law 03-573, called the Privacy Act of 1974, requires that you be informed of the purpose

and uses to be made of the information that is collected.

The Department of the Army may collect the information requested under authority of 10 United

States Code 137.

Providing information in this questionnaire is voluntary. Failure to respond to any particular

question will not result in any penalty for the respondent.

The information collected in this survey will be used to evaluate military intelligence systems and

their utilization.

The information will be used for research and analysis only. The US Army Test and Evaluation

Command (ATEC) has primary research, analysis, and evaluation responsibility.

Instructions:

1. On Page 2:

a) Check the Functional Identities on which you are serving as an Operator

b) Indicate your Echelon (e.g., Btn, Bde, Div) of operation (it is understood that some soldiers are operating in more

than one functionality)

c) Indicate name of your workstation--keyboard and monitor

2. On Page 3 please read the descriptions of the seven possible responses. For each of the questions

beginning on Page 4, read each question carefully and circle the response that appropriately reflects your

opinion. Although not required, we would welcome any written comment you may wish to provide in

response to a particular question. Please use the reverse of the page for these entries and key your

responses to the question number.
49

BIO/BACKGROUND/POSITION

Name ___________________ PIN No. Date ______________

(Please Print) (Last 4 SSN) MM/DD/YY

a) Functional Identities (ACE Components) on which you have been serving as a Test Participant

Operator (please check):

__ ISS

__CCS

__ Shared SS

__ COMINT

__ IMINT

__ RIS/ELINT

__ SIGINT/MASINT

__ CI/HUMINT

__ OSINT

__ AS

__SIT/IPB/DB

__ TGTDEV

__ CM/ISR

__Asset Mgmnt

__ Trusted Suite (TWS)

b) Echelon (Btn, Bde, Div)_________________

c) Workstation________________ Keyboard________________ Monitor___________________
50

Rating Scheme: The rating criteria used for the following questions are defined below. Please ensure that you

understand these rating criteria when answering questions with a rating scheme.

1: Completely Agree: There must be absolutely no doubt when using this response that the item being evaluated

cannot be any better designed, written, or has great value and is a desirable system feature or function.

2: Strongly Agree: This response indicates that the item being evaluated is very good and very helpful to the

analyst/operator.

3: Generally Agree: This response indicates the item being evaluated is acceptable and helpful to the

analyst/operator.

4: Generally Disagree: This response indicates the item being evaluated is unacceptable, but only minor

improvements are required before it is acceptable. Written comment indicating examples of how change should be

made would be appropriate when this rating is used. Please write comments on back of form and reference item no.

5: Strongly Disagree: This response indicates that the item being evaluated is unacceptable and major

improvements are required to make it acceptable. Written comment indicating examples of how change should be

made would be appropriate when this rating is used. Please write comments on back of form and reference item no.

6: Completely Disagree: There must be absolutely no doubt when using this response that the item being

evaluated is of unacceptable design, composition, or value and must be completely redesigned, rewritten, or

modified to be acceptable. Written comment indicating examples of how change should be made would be

appropriate when this rating is used. Please write comments on back of form and reference item no.

0: Unknown: This response indicates that you do not have sufficient knowledge to answer the question or you

have not experienced the action or item in question.
51

HUMAN FACTORS ENGINEERING QUESTIONNAIRE

1 = Completely Agree 3 = Generally Agree 5 = Strongly Disagree 0 = No knowledge

2 = Strongly Agree 4 = Generally Disagree 6 = Completely Disagree or Experience

Environmental/Physical Concerns

(1) The brightness level on the computer screens can be adjusted to a

comfortable level. 1 2 3 4 5 6 0

(2) Eye strain at the end of the shift doesnotseemtobeaproblem.1 2 3 4 5 6 0

(a) Video displays are free of flicker.1 2 3 4 5 6 0

(b) Video displays allow good letter discrimination.1 2 3 4 5 6 0

(c) Video displays viewing distance isacceptable.1 2 3 4 5 6 0

(d) Video displays angle of view is acceptable.1 2 3 4 5 6 0

(e) Location of video display is acceptable.1 2 3 4 5 6 0

(f) Adjustability of video displays isacceptable.1 2 3 4 5 6 0

(3) The workstation requires a “glare screen”.1 2 3 4 5 6 0

(4) Operating the system components does not cause any physical discomfort

(such as, back strain and arm or wriststrain).

1 2 3 4 5 6 0

(a) Size of keyboard and controls is acceptableforeffectiveuse.1 2 3 4 5 6 0

(b) Shape of keyboard and controls isacceptableforeffectiveuse.1 2 3 4 5 6 0

(c) Spacing between controls on keyboardisacceptableforeffectiveuse.1 2 3 4 5 6 0

(d) Resistance (too easy to turn or push or too hard to turn or push) is

acceptable for effective use.

1 2 3 4 5 6 0

(e) Keyboard and controls are correctlylabeled.1 2 3 4 5 6 0

(f) Keyboard and controls are the correctsizeforeasyvisibility.1 2 3 4 5 6 0

(g) Keyboard and controls are easy tounderstand.1 2 3 4 5 6 0

(h) Keyboard and controls do not haveanyunrelatedorconfusingmarkings.1 2 3 4 5 6 0

(i) Keyboard and controls are easy toseeandareclearlyvisible.1 2 3 4 5 6 0

(j) Keyboard and controls are at the rightangleofview.1 2 3 4 5 6 0

(k) Keyboard and controls are easily identifiable.1 2 3 4 5 6 0

(l) Keyboard and critical controls arewithineasyreach.1 2 3 4 5 6 0

(m) Keyboard and non-critical controlsisacceptable.1 2 3 4 5 6 0

(n) Keyboard and non-critical controlsarewithineasyreach.1 2 3 4 5 6 0

(o) Keyboard and controls are functionallygroupedtogether.1 2 3 4 5 6 0

(p) Keyboard and control types are correctforthefunctiontheyperform.1 2 3 4 5 6 0

(q) Keyboard and controls aural controlsandwarningsareeasytohear.1 2 3 4 5 6 0

(5) Dust covers are required on key components of the system (keyboard,

workstation). 1 2 3 4 5 6 0

Software

(6) It is easy to delete unneeded data from the display screen while performing

procedures or tasks. 1 2 3 4 5 6 0

(7) It is easy to access menus. 1 2 3 4 5 6 0

(8) The unfamiliar terms or commands are defined on the HELP screens. 1 2 3 4 5 6 0

(9) Help prompts are always available in a display or are provided in response

to a HELP request to assist in accomplishingthetask.1 2 3 4 5 6 0
52

1 = Completely Agree 3 = Generally Agree 5 = Strongly Disagree 0 = No knowledge

2 = Strongly Agree 4 = Generally Disagree 6 = Completely Disagree or Experience

(10) The number and sequence of steps required to accomplish a task are

logical and follow an orderly path. 1 2 3 4 5 6 0

(11) All keystrokes/actions performed from one menu to the next are

accomplished usingthe same seriesofkeys/commands.1 2 3 4 5 6 0

(12) All prompts or messages appear in the same position and are in the same

format. 1 2 3 4 5 6 0

(13) At the completion of a task, the computer provides a prompt or message

that shows the actions are completed.1 2 3 4 5 6 0

(14) While actions are being processed, the computer displays an “ICON”

showing that the transaction is in process.1 2 3 4 5 6 0

(15) Response to menu selections, keypad operations, and graphic operations

are immediate. 1 2 3 4 5 6 0

(16) The cursor location is easy to find. 1 2 3 4 5 6 0

(17a) The cursor can be moved easily and accurately to any location on the

screen (using the built-in mouse pad).1 2 3 4 5 6 0

(17b) The cursor can be moved easily and accurately to any location on the

screen (using the external mouse).1 2 3 4 5 6 0

(18) It is easy return to previous steps in an operation to correct an error or to

make other changes. 1 2 3 4 5 6 0

(19) It is easy to exit from a current operation or function. 1 2 3 4 5 6 0

(20) It is easy to cancel changes and restore the previous display. 1 2 3 4 5 6 0

(21) Mandatory and optional data fields are easily distinguished from each

other. 1 2 3 4 5 6 0

(22) When an entry in an optional field makes other normally optional fields

mandatory, the computer provides sufficientpromptstoalerttheoperator.1 2 3 4 5 6 0

(23) When mandatory data entries are omitted, a “warning message” alerts the

operator to provide the required information.1 2 3 4 5 6 0

(24) Error messages are easy to understand. 1 2 3 4 5 6 0

(25) Prompts are always provided prior to overwriting or exiting the

message/file to avoid loss of data. 1 2 3 4 5 6 0

(26) An UNDO command is available to reverse actions. 1 2 3 4 5 6 0

(27) Menus can be arranged so they do not interfere with the accomplishment

of tasks. 1 2 3 4 5 6 0

(28) Frequently accessed menus are consistent in format and functionality from

one to another. 1 2 3 4 5 6 0

(29) It is easy to correct message errors character by character, line by line, and

field by field. 1 2 3 4 5 6 0
53

1 = Completely Agree 3 = Generally Agree 5 = Strongly Disagree 0 = No knowledge

2 = Strongly Agree 4 = Generally Disagree 6 = Completely Disagree or Experience

(30) Data fields are grouped and ordered logically way which made message

building easy. 1 2 3 4 5 6 0

(31) All data field labels are easily understood. 1 2 3 4 5 6 0

(32) All data field instructions are easily understood. 1 2 3 4 5 6 0

(33) It was easy to return to the main menu or base-level functions. 1 2 3 4 5 6 0

(34) Menu options defined the function to be performed. 1 2 3 4 5 6 0

(35) The menu structure and design permits immediate access to critical or

frequently selected options. 1 2 3 4 5 6 0

(36) The current position within the menu structure was always known. 1 2 3 4 5 6 0

(37) Menu options use familiar terminology and each item is distinct from the

other. 1 2 3 4 5 6 0
54

2.3.1.2.7 Human Factors Engineering (Function-Specific) Questionnaire (table 11). This 

instrument—referenced in row 2, column 1 on the second page of table 4—was designed with 

the assistance of New Systems Training Integration Office (NSTIO) staff from Fort Huachuca 

and is designed to address function-specific software problems. The specific form presented 

here (see page 55) was used to support the Block 2 IOTE testing for ASAS. Development of this 

instrument utilized the Enabling and Terminal Learning Objectives found in the Lesson Plans 

used to train soldiers in their specific functionalities. 

Appendix H provides a rather comprehensive Human Factors Engineering (Function-Specific) 

Questionnaire/Interview instrument used during the Block 2 ASAS IOTE in March 2005. This 

instrument asks questions for each of the participating Analysis and Control Element (ACE) 

Functional Identities and judgments about potential MANPRINT problems from both an 

Equipment and Function perspective. Soldiers were asked to indicate whether problems existed 

and to provide ratings about how well they agreed with the specific potential problems that may 

have existed. Ratings are sometimes helpful as descriptive statistics to tease out the importance 

of the reported problems; however, because the number of soldiers participating in each system 

function (Functional Identity) is generally very small, those ratings rarely are amenable to any 

inferential statistical analyses. While soldiers were asked to explain problems by keying their 

written response to specific questions, they frequently did not. In spite of very strong repeated 

urging of testing management to permit follow-up interviews with each soldier to clarify 

unanswered questions (or questions whose answers were ambiguous), this opportunity was 

denied. With review of ratings and responses of each participant to each form, interviews can be 

conducted to clarify responses only to items evidencing ambiguities needing fuller explanation. 
55

HUMAN FACTORS ENGINEERING (FUNCTION-SPECIFIC) QUESTIONNAIRE 

FOR THE SINGLE SOURCE WORK STATIONS 

BLOCK 2--INITIAL OPERATIONAL TEST AND EVALUATION (IOTE)2

Purpose: This questionnaire is designed to capture data about the ability of the soldier to use the 

Single Source (SS) Work Station to accomplish the tasks of COMINT, IMINT HUMINT, and 

OSINT processing and dissemination. The data you provide will help decision makers evaluate 

(and correct, if necessary) how effectively the Single Source Work Stations and associated 

software have been designed to assist and enhance production and dissemination of intelligence 

products. None of your individual data will be provided to your chain of command nor entered 

into your personnel files. Your responses will be maintained under strict enforcement of 

Confidentiality Public Law. 

Instructions: Read each item carefully and circle the appropriate numbered response. The 

response must reflect your perception of how well the Single Source Work Stations have assisted 

and enhanced your ability to perform COMINT, IMINT HUMINT, and OSINT processing and 

dissemination. The questionnaire is broad in scope; therefore, some questions may not pertain to 

the specific functionality you are performing. In those cases, circle “N/A = 0”. If you Disagree 

with any statement (ratings 4, 5, or 6), an explanation “why” is required. Please write your 

response on the back of this form and indicate for which question you are making comment. 

Items whose number is indented are sub-functions or tasks required to perform the function 

whose number is not indented. 

BIO/BACKGROUND/POSITION

Name ______________________________ PIN No. ______________ Date ____________ 

 (Please Print) (Last 4 SSN) MM/DD/YY 

Equipment/System (circle or indicate): 

CI/HUMINT Work Station 

OSINT Work Station 

IMINT Work Station 

COMINT Work Station 

 

Test Phase: Evaluation Window 2--Mid Test 

 2

Functions, sub-functions and tasks presented in this survey have been provided through review of the Enabling and Terminal 

Learning Objectives found in the Lesson Plans used to train soldiers of the ACE Functionalities. They have been provided by the 

New Systems Training and Integration Office (NSTIO), Fort Huachuca, AZ. 
56

The rating scheme is as follows: 

1: Completely Agree: There must be absolutely no doubt when using this response that the item being evaluated 

cannot be any better than it currently is and is a desirable design feature. 

2: Strongly Agree: This response indicates that the item being evaluated is very good and very helpful to the 

analyst/operator. 

3: Generally Agree: This response indicates the item being evaluated is acceptable and helpful to the 

analyst/operator. 

4: Generally Disagree: This response indicates the item being evaluated is unacceptable, but only minor 

improvements are required before it is acceptable. (Please indicate specifically what improvements need to be 

made.) 

5: Strongly Disagree: This response indicates that the item being evaluated is unacceptable and major 

improvements are required to make it acceptable. (Please indicate specifically what improvements need to be 

made.) 

6: Completely Disagree: There must be absolutely no doubt when using this response that the item being 

evaluated is unacceptable and must be completely redesigned or rewritten to be acceptable. (Please indicate 

specifically what improvements need to be made.) 

0: Unknown: This response indicates that you do not have sufficient knowledge to answer the question or you 

have not experienced the action or item in question. 
57

Based on your experience, rate your ability to perform the following functions with the Single Source Work Station: 

For The OSINT (questions 1-36) 

1.) The Single Source Work Station works well to perform the OSINT functions. 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

Processing with OSINT assisted in performing the following operations: 

 2.) EMPLOYING THE OPEN SOURCE AUTOMATED LINK ANALYSIS TOOL (OSALAT) ON THE 

UNCLASSIFIED OSINT COLLECTOR WORKSTATION (OCW) 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 3.) Starting the Local Task Server 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

4.) Creating a Case File 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 5.) Opening a Case File 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 6.) Creating a Search Scenario 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 7.) Searching for Information in OSALAT 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 8.) Reviewing retrieved documents in the Document Editor 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 9.) Exporting Case Data 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 10.) CONFIGURING THE OSINT EXPORT TOOL 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 11.) Opening the Output Queue 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 12.) Opening the Export Tool 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 13.) Configuring the OSINT Export Scheduler 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 14.) Setting Export Type to Manual Updates 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

15.) EMPLOYING THE INDIVIDUAL, EVENTS, AND ORGANIZATIONS (IE&O) DATABASE 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 16.) Reviewing IE&O Data in the Entity Editor 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0
58

 17.) Creating a New Case 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 18.) Importing External Data into the Entity Editor 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 19.) Creating New Individual 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 20.) Adding a New Attribute to an Individual 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 21.) Adding an Event to the Case 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 22.) Adding a New Attribute to an Event 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 23.) Adding an Organization to the Case 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 24.) Adding a New Attribute to an Organization 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 25.) Adding Equipment to the Case 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 26.) Adding a New Attribute to a piece of Equipment 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 27.) Adding a Facility to the Case 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 28.) Adding a New Attribute to a Facility 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 29.) Adding a Unit to the Case 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 30.) Adding a New Attribute to a Unit 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 31.) Adding a Place to the Case 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 32.) Adding a New Attribute to a Place 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 33.) Creating Relationships in the Case 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 
59

 34.) Adding Attributes to an Entity 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 35.) Publishing from Entity Editor to TMB 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 36.) Plotting from the Entity Editor to the Map 

 Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

For the IMINT Section (37-47) 

37.) The Single Source Work Station works well to perform the IMINT functions. 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

Processing with IMINT assisted in performing the following operations: 

 38.) PERFORMING IMAGERY OPERATIONS ON THE MIP 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 39.) Viewing an Image using Imagery Explorer (IE) 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 40.) Adding annotations to ELT 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 41.) Geo-registering and converting an image 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 42.) Exporting an image to the JMTK map (Draping) 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 43.) Placing an Image Icon on a map for display 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 44.) Pushing an Image from MIP to the ISS Multimedia Journal 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 45.) Pushing an Image from MIP to a Specific USER (ASWS) on the ISS 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 46.) Pulling Imagery from ISS to MIP 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 47.) Pulling Products from IPL using JWICS 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

For the COMINT Section (48-100) 

48.) The Single Source Work Station works well to perform the COMINT functions. 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0
60

Processing with COMINT assisted in performing the following operations: 

 49.) OPERATING THE MASINT-AT INTERACTIVE CORRELATOR 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 50.) Performing a PDR query 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 51.) Creating a CDR from PDR(s) 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 

 52.) Performing a Candidate Search 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 53.) Correlating PDR to CDR 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 54.) Filtering Data Records 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 55.) Merging CDR’s 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

56.) CONFIGURING THE COMINT EXPORT GATEWAY 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

57.) PERFORMING REFERENCE DATABASE OPERATIONS 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

58.) QUERYING INTELLIGENCE SHARED SERVER DATABASES VIA THE WEB 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

59.) Querying database records on the ISS 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

60.) Viewing queried records 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

61.) Sending selected data records to the Map on the ISS 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

62.) MANAGING COMINT ALERTS 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

63.) CORRECTING ERRED COMINT MESSAGES 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0
61

 64.) Identifying an erred SIGINT message 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 65.) Correcting erred SIGINT messages 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

66.) Deleting a bad SIGINT message 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

67.) Opening the COMINT Export Gateway 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

68.) Setting the update frequency 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 69.) Configuring the gateway output queue 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 70.) Performing manual data transfer functions 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 71.) Clearing the Export Gateway log 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 

 72.) Exiting the Export Gateway 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

73.) MANAGING COMINT CORRELATION RULE SETS 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

74.) OPERATING THE COMINT ANALYSIS TOOLSET 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 75.) Observing COMINT Case Explorer functions 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 76.) Observing COMINT-AT Editor functions 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 77.) Creating a Case folder 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 78.) Using Case Management features in the COMINT-AT Case Explorer 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0
62

 79.) Setting field views in the COMINT-AT Editor 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 80.) Plotting COMINT data to the map from the COMINT-AT Editor 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 81.) Generating a U.S. Message Text Formatting (USMTF) message from COMINT data 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 82.) Viewing data record source messages 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 83.) Creating a Parsed Data Record (PDR) manually 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 84.) Using the COMINT Case Explorer Search tool to search for folder content 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 85.) Searching for Network, CDR, or PDR COMINT data records using the COMINT 

 Case Explorer Search tool 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 86.) Performing manual correlation functions 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 87.) Performing Line of Bearing resolution 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 88.) Stopping, Starting, and Pausing the Correlator Service 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 89.) Sending selected Case Explorer products to TMB Folders 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 90.) Archiving PDRs 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 91.) Restoring archived PDRs 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

92.) OPERATING THE COMINT-AT GIST TOOL 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

93.) OPERATING THE COMINT-AT CRYPTANALYSIS TOOL 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0
63

94.) USING COMINT VISUALIZATION TOOLS 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 95.) Viewing network visualization (network diagram) 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 96.) Editing network visualization (network diagram) 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 97.) Sending a USMTF message from COMINT visualization 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 98.) Viewing Order of Battle visualization 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 99.) Publishing COMINT visualizations to Microsoft PowerPoint 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

100.) PERFORMING NEAR REAL TIME INTELLIGENCE OPERATIONS 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

For the HUMINT section (101-138) 

101.) The Single Source Work Station works well to perform the HUMINT functions. 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

Processing with HUMINT assisted in performing the following operations: 

 

102.) PERFORMING TWS, WEBGUARD AND WEBSHIELD OPERATIONS 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 103.) Verifying proxy settings for WebShield operations 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 104.) Retrieving a collateral File via WebShield 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

105.) QUERYING THE DATABASE VIA THE WEB 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 106.) Querying database records on the ISS 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 107.) Viewing queried records 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0
64

 108.) Viewing Association, multimedia, or Cross Links for data in the Query Results window 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

109.) PERFORMING FILE TRANSFER PROTOCOL (FTP) OPERATIONS 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 

 110.) Initiating a new FTP session 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 111.) Setting initial directories for a selected profile 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 112) Setting advanced settings 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

113.) CONFIGURING CHAMS ACE BLOCK II SYSTEM SETTINGS 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 114.) Configuring CHAMS System Settings 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

115.) PREPARING THE CI/HUMINT WORKSTATION FOR OPERATIONS (CF-73 LAPTOP) 

 WITHIN THE ACE BLOCK II CONFIGURATION 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 116.) Unpacking the laptop components from the carrying case 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 117.) Assembling the computer 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 118.) Attaching other devices 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 119.) Applying power to the system 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

120.) CONFIGURING CHAMS ACE BLOCK II COMMUNICATIONS 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 121.) Configuring Windows 2000 Communications 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 122.) Configuring the CHAMS Workstation to the ACE Classified Printer 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0
65

 123.) Configuring the CHAMS Address Book 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

124.) PERFORMING CHAMS REPORT MANAGER FUNCTIONS FOR ACE OPERATIONS 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 125.) Accessing Report Manager 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 126.) Creating a CI/HUMINT Report for ACE Transfer 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 127.) Queuing Reports for Correlation 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 128.) Sending Reports into the ISS 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 129.) Deleting Reports from the ACE ISS 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 130.) Editing a CI/HUMINT Report 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 131.) Plotting a CI/HUMINT Report 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

132.) PREPARING THE ACE CI/HUMINT WORKSTATION FOR MOVEMENT 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 133.) Terminating Operations 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 134.) Disassembling the system 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

 135.) Storing the system 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

136.) ESTABLISHING AN ACE ADDRESS BOOK 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

137.) CONFIGURING THE ISS CHAMS WEB LINK INTERFACE 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0 

138.) ESTABLISHING ISS DATA LINK USING THE WEB INTERFACE 

Completely Agree 1 2 3 4 5 6 Completely Disagree N/A=0
66

2.3.1.2.8 Documentation of Inter-Operability Difficulties--Problems Receiving/Sending 

Different Message Types Among (Intra/Inter) Battlefield Functional Areas (BFA) (table 7). This 

form—referenced in row 3, column 1 on the second page of table 4, Documentation of InterOperability—provides a rather comprehensive opportunity for soldiers to report difficulties 

receiving and sending different message types among the participating Functional Identities and 

for identifying both intra/inter-communication problems among the BFA. This form has been 

used in many of the ASAS sub-system LUTEs as well as the block 2 IOTE. 
67

DOCUMENTATION OF INTEROPERABILITY PROBLEMS 

DURING ANALYSIS AND CONTROL ELEMENT (ACE) 

INITIAL OPERATIONAL TEST AND EVALUATION (IOTE)

BLOCK 2 

BIO/BACKGROUND/POSITION

Name PINno. Date Time ____________ 

 Please Print Mo/Da/Yr 0000-2400 

Please circle ACE position (Functional Identity): 

CCS SMART Multifunction Work Station RIS/ELINT SS AS SIT/IPB DBM 

SIGINT Trusted Suite ISR CI/HUMINT OSINT 

Please circle echelon: 

BTN DIV BDE CORP EAC 

MOPP(Please circle): 0 1 2 3 4 

INSTRUCTIONS

For each of the listed MESSAGE TYPES, please indicate: 

 whether you had PROBLEM RECEIVING 

 FROM WHOM was the message being sent (if known) 

 BRIEF DESCRIPTION of problem 

 whether you had PROBLEM SENDING 

 TO WHOM were you sending the message 

 BRIEF DESCRIPTION of problem 
68

Table 7. Documentation of inter-operability difficulties—problems receiving/sending different message types among (intra/inter) battl

(BFA). 

NAME ___________

INTEROPERABILITY PROBLEMS 

a. 

MESSAGE TYPE 

b. 

PROBLEM 

RECEIVING 

(check if 

YES) 

c. 

FROM 

WHOM 

(see 

below) 

d. 

BRIEF DESCRIPTION 

(use reverse side if necessary) 

e. PROBLEM 

SENDING 

(check if YES) 

f. 

TO WHOM 

(see below) 

(u

1. Tactical Reports (TACREP) (C111) 2. Size, Activity, Location, Unit, Time, and  Equipment Report (SALUTE) (S303) 3. Intelligence Report (INTREP) (C110) 4. Artillery Target Intelligence. Target  Criteria Report (ATI.TCRIT) (D281) 5. Geometry Message (S201) 6. Imagery Interpretation Report (IIR) 7. Reconnaissance Exploitation Report  (RECCEXREP) (C101) 8. Radar Exploitation Report (REXREP)  (X031) 9. Intelligence Summary (INTSUM) 10. Order Message (A423) 

c. FROM WHOM: f. TO WHOM: 

DIVISION LEVEL: 1) RWS 2) TOC 3) MSE 4) CNR 5) NTDR 6) SINCGARS 7) EPLRS 8) AUTODIN 9) TROJAN SPIRIT 10) CGS 11) UAV 12) MCS 13) AFATDS 14) CSSALL SOURCE 17) CHATS 18) IMETS 19) DTSS 20) ACT-E 21) FBCB2 22) AMDWS 

BRIGADE LEVEL: 23) RWS 24) TOC 25) MSE 26)CNR 27) NTDR 28) SINCGARS 29) EPLRS 30) AUTODIN 31) TROJAN SPIRIT 32) CGS 33) UAV 34) MCS 35) AFATD

38) ASAS ALL SOURCE 39) CHATS 40) IMETS 41) DTSS 42) ACT-E 43) FBCB2 44) AMDWS 
69

Table 7. Documentation of inter-operability difficulties—problems receiving/sending different message types among (intra/inter) battl

(BFA) (continued). 

a. 

MESSAGE TYPE 

b. 

PROBLEM 

RECEIVING 

(check if 

YES) 

c. 

FROM 

WHOM 

(see 

below) 

d. 

BRIEF DESCRIPTION 

(use reverse side if necessary) 

e. 

PROBLEM 

SENDING 

(check if 

YES) 

f. 

TO 

WHOM 

(see below) 

(

11. Operations Plan or Order Change Message  (PLANORDCHG) 12. Request for Information (RI) (F014) 13. Response to Request for Information  (RRI) (F015) 14. Free-text Message (FREETEXT) (S302) 15. Tactical Electronic Intelligence Report  (TACELINT) (C121) 16. Weather Forecast Message (WXFCST) 17. Multiple Asset Status Report (MASTR)  (S304) 18. External Database Coordination (EDC)  (C110M) 19. Variable Message Format Message (VMF) 

c. FROM WHOM: f. TO WHOM: 

DIVISION LEVEL: 1) RWS 2) TOC 3) MSE 4) CNR 5) NTDR 6) SINCGARS 7) EPLRS 8) AUTODIN 9) TROJAN SPIRIT 10) CGS 11) UAV 12) MCS 13) AFATDS 14) CSS

ALL SOURCE 17) CHATS 18) IMETS 19) DTSS 20) ACT-E 21) FBCB2 22) AMDWS 

BRIGADE LEVEL: 23) RWS 24) TOC 25) MSE 26)CNR 27) NTDR 28) SINCGARS 29) EPLRS 30) AUTODIN 31) TROJAN SPIRIT 32) CGS 33) UAV 34) MCS 35) AFATDS 

38) ASAS ALL SOURCE 39) CHATS 40) IMETS 41) DTSS 42) ACT-E 43) FBCB2 44) AMDWS 
70

Table 7. Documentation of inter-operability difficulties—problems receiving/sending different message types among (intra/inter) battl

(BFA) (continued). 

a. 

MESSAGE TYPE 

b. PROBLEM RECEIVING (check if YES) 

c. 

FROM 

WHOM 

(see 

below) 

d. 

BRIEF DESCRIPTION 

(use reverse side if necessary) 

e. PROBLEM SENDING (check if YES) 

f. 

TO 

WHOM 

(see below) 

g. 

(u

20. TIDAT (S305) 21. ATLIEWTC (S308) 22. MSGCHGREP (C001) 23. AFU.MFR (C241) 24. ATLATR (C281) 25. RESOURCES (S507L) 26. MAER (S301) 27. MATM (X014) 28. STOPJAM (S307) 29. OBSERVED POSITION REPORT  (K05.52) 

c. FROM WHOM: f. TO WHOM: 

DIVISION LEVEL: 1) RWS 2) TOC 3) MSE 4) CNR 5) NTDR 6) SINCGARS 7) EPLRS 8) AUTODIN 9) TROJAN SPIRIT 10) CGS 11) UAV 12) MCS 13) AFATDS 14) CSSC

ALL SOURCE 17) CHATS 18) IMETS 19) DTSS 20) ACT-E 21) FBCB2 22) AMDWS 

BRIGADE LEVEL: 23) RWS 24) TOC 25) MSE 26)CNR 27) NTDR 28) SINCGARS 29) EPLRS 30) AUTODIN 31) TROJAN SPIRIT 32) CGS 33) UAV 34) MCS 35) AFATD

38) ASAS ALL SOURCE 39) CHATS 40) IMETS 41) DTSS 42) ACT-E 43) FBCB2 44) AMDWS 
71

Table 7. Documentation of inter-operability difficulties—problems receiving/sending different message types among (intra/inter) battl

(BFA) (continued). 

a. 

MESSAGE TYPE 

b. PROBLEM RECEIVING (check if YES) 

c. 

FROM 

WHOM 

(see 

below) 

d. 

BRIEF DESCRIPTION 

(use reverse side if necessary) 

e. PROBLEM SENDING (check if YES) 

f. 

TO 

WHOM 

(see 

below) 

30. FREETEXT (K01.1) 31. SPOT/SALUTE (K04.52) 32. OVERLAY (K0.52) 33. SCT digital picture (JPEG) 34. RELEVANT COMMON PICTURE  (MCO) 35. SID (NITF) 36. DIGITAL PICTURE (NITF) 37. Other (specify) 

c. FROM WHOM: f. TO WHOM: 

DIVISION LEVEL: 1) RWS 2) TOC 3) MSE 4) CNR 5) NTDR 6) SINCGARS 7) EPLRS 8) AUTODIN 9) TROJAN SPIRIT 10) CGS 11) UAV 12) MCS 13) AFATDS 14) CSS

ALL SOURCE 17) CHATS 18) IMETS 19) DTSS 20) ACT-E 21) FBCB2 22) AMDWS 

BRIGADE LEVEL: 23) RWS 24) TOC 25) MSE 26)CNR 27) NTDR 28) SINCGARS 29) EPLRS 30) AUTODIN 31) TROJAN SPIRIT 32) CGS 33) UAV 34) MCS 35) AFATDS 

38) ASAS ALL SOURCE 39) CHATS 40) IMETS 41) DTSS 42) ACT-E 43) FBCB2 44) AMDWS 
72

2.3.1.2.9 Final Debrief—Analyst/Data Collector. These forms—referenced in rows 4 and 5, 

column 1 on the second page of table 4—provide an opportunity for analysts and data collectors, 

respectively, to provide a final documentation of problems reported on the system. Questions 

asked (see pages 73 and 78) tend to be broad in nature concerning the functionalities operational 

capability and the capability of some of the major parameters needed to navigate through the 

software. 
73

FINAL DEBRIEF

ASAS SINGLE SOURCE OPERATOR/ANALYST QUESTIONNAIRE 

Purpose: This questionnaire is used to capture your views concerning the operational aspects 

and utility of ASAS. The data you provide will help decision makers evaluate how effective the 

ASAS has been designed to assist you in the performance of your intelligence gathering, 

processing and analysis tasks. 

PRIVACY ACT STATEMENT 

 Public Law 93-573, called the Privacy Act of 1974, requires that you be informed of the purpose and uses to 

be made of the information that is collected. 

 The Department of the Army may collect the information requested under authority of 10 United States 

Code 137. 

 Providing information in this questionnaire is voluntary. Failure to respond to any particular question will 

not result in any penalty for the respondent. 

 The information collected in this survey will be used to evaluate military intelligence systems and their 

utilization. 

 The information will be used for research and analysis only. The US Army Operational Evaluation 

Command has primary research, analysis, and evaluation responsibility. 

Instructions: First, read the descriptions of the seven possible responses and then read each 

question carefully and circle the response that appropriately reflects your opinion. Although not 

required, we would welcome any written comment you may provide in response to a particular 

question. Please use the reverse of the page for these entries and key your responses to the 

question number. 

GENERAL INFORMATION. 

Name/Rank:_________________________ Position Observed: __________________ 
74

Rating Scheme: The rating criteria used for the following questions are defined below. Please 

ensure that you understand these rating criteria when answering questions with a rating scheme. 

1: Completely Disagree: There must be absolutely no doubt when using this response that 

the item being evaluated is of unacceptable design, composition, or value and must be 

completely redesigned, rewritten, or modified to be acceptable. 

2: Strongly Disagree: This response indicates that the item being evaluated is unacceptable 

and major improvements are required to make it acceptable. 

3: Generally Disagree: This response indicates the item being evaluated is unacceptable, 

but only minor improvements are required before it is acceptable. 

4: Generally Agree: This response indicates the item being evaluated is acceptable and 

helpful to the analyst/operator. 

5: Strongly Agree: This response indicates that the item being evaluated is very good and 

very helpful to the analyst/operator. 

6: Completely Agree: There must be absolutely no doubt when using this response that the 

item being evaluated cannot be any better designed, written, or has great value and is a desirable 

system feature or function. 

9: Unknown: This response indicates that you do not have sufficient knowledge to answer 

the question or you have not experienced the action or item in question. 
1 = Completely Disagree 3 = Generally Disagree 5 = Strongly Agree 9 = Unknown 

2 = Strongly Disagree 4 = Generally Agree 6 = Completely Agree 

75

SINGLE SOURCE OPERATOR/ANALYST QUESTIONNAIRE 

(1) I am confident in my ability to properly populate, store, and maintain Single Source 

databases. 

 1 2 3 4 5 6 9 

 

(2) I am confident in the accuracy of the information contained in the Single Source 

databases. 

 1 2 3 4 5 6 9 

 

(3) ASAS significantly helped me do my job as a COMINT Analyst. 1 2 3 4 5 6 9 

 

(4) ASAS assisted me in producing a COMINT picture of the battlefield that was 

complete. 

 1 2 3 4 5 6 9 

 

(5) ASAS assisted me in producing a COMINT picture of the battlefield that was 

accurate. 

 1 2 3 4 5 6 9 

 

(6) ASAS assisted me in producing a COMINT picture of the battlefield that was timely. 1 2 3 4 5 6 9 

 

(7) ASAS significantly helped me do my job as an ELINT Analyst. 1 2 3 4 5 6 9 

 

(8) ASAS assisted me in producing an ELINT picture of the battlefield that was 

complete. 

 1 2 3 4 5 6 9 

 

(9) ASAS assisted the analyst observed in producing an ELINT picture of the battlefield 

that was accurate. 

 1 2 3 4 5 6 9 

 

(10) ASAS assisted me in producing an ELINT picture of the battlefield that was timely. 1 2 3 4 5 6 9 

 

(11) As the Target Analyst, I was able to significantly contribute to the ACE targeting 

process by identifying high payoff targets. 

 1 2 3 4 5 6 9 

 

(12) The time to develop, run, and display queries against ASAS databases does not 

adversely effect analysis. 

 1 2 3 4 5 6 9 

 

(13) The alarm process worked well. 1 2 3 4 5 6 9 

 

(14) Single Source graphics tools enabled me to display and evaluate the effects that 

terrain and mobility factors may have on enemy courses of action. 

 1 2 3 4 5 6 9 

 

(15) The ability to automatically generate TACREPS/TACELINTS through the Single 

Source TACREP generator was effective. 

 1 2 3 4 5 6 9 

 

(16) The Single Source leadership is fully aware of the capabilities of ASAS. 1 2 3 4 5 6 9 

 

(17) The Single Source leadership is aware of the required interaction with the all source 

leadership to support ACE operations. 

 1 2 3 4 5 6 9 

 
1 = Completely Disagree 3 = Generally Disagree 5 = Strongly Agree 9 = Unknown 

2 = Strongly Disagree 4 = Generally Agree 6 = Completely Agree 

76

(18) Based on my observations and interactions with other Single Source Analysts, we 

were able to effectively contribute to answering PIRs. 

 1 2 3 4 5 6 9 

 

(19) ADP operations personnel appeared to be always able to identify the source of 

software problem. 

 1 2 3 4 5 6 9 

 

(20) ADP operations personnel appeared to be always able to access the command line 

and immediately fix software problems. 

 1 2 3 4 5 6 9 

 

(21) Single Source systems administrator requires access to system software through the 

command line. 

 1 2 3 4 5 6 9 

 

(22) ASAS operations would be significantly improved if a designated operator/analyst 

was trained in UNIX and had command line access. 

 1 2 3 4 5 6 9 

 

(23) ASAS enabled me to handle the volume of messages received during the LUT. 1 2 3 4 5 6 9 

 

(24) The inability to maintain continual communication with external systems impacted 

on my ability to conduct Single Source analysis. 

 1 2 3 4 5 6 9 

 

(25) The Single Source enclave was easily displaced in MOPP 0. 1 2 3 4 5 6 9 

 

(26) The Single Source enclave was easily emplaced in MOPP 0. 1 2 3 4 5 6 9 

 

(27) The Single Source enclave was easily displaced in MOPP IV. 1 2 3 4 5 6 9 

 

(28) The Single Source enclave was easily emplaced in MOPP IV. 1 2 3 4 5 6 9 

 

(29) I was able to accomplish analytical tasks using ASAS while operating in MOPP IV. 1 2 3 4 5 6 9 

 

(30) Software failures during the tactical phase of the LUT did not cause a loss of 

confidence in the system. 

 1 2 3 4 5 6 9 

 

(31) The majority of the software problems can be traced to database permissions being 

improperly set. 

 1 2 3 4 5 6 9 

 

(32) When a hardware problem occurred, military maintenance personnel responded in a 

timely manner. 

 1 2 3 4 5 6 9 

 

(33) When a hardware problem occurred, military maintenance personnel isolated the 

problem quickly. 

 1 2 3 4 5 6 9 

 

(34) When a hardware problem occurred, military maintenance personnel were able to 

fix the problem without difficulty. 

 1 2 3 4 5 6 9 

 

(35) ASL and PLL allocations at organic level appear to be sufficient. Downtime due to 

lack of spare parts was not experienced. 

 1 2 3 4 5 6 9 
1 = Completely Disagree 3 = Generally Disagree 5 = Strongly Agree 9 = Unknown 

2 = Strongly Disagree 4 = Generally Agree 6 = Completely Agree 

77

(36) Operating under ACE Concepts and Doctrine enhances the ability to provide 

complete, timely, and accurate intelligence for tactical decision making by the G3 or 

operational staff. 

 1 2 3 4 5 6 9 

 

(37) Sufficient personnel were available to operate ASAS on a routine basis. 1 2 3 4 5 6 9 

 

(38) The ability to pass graphic formatted information to outside the ACE is critical to 

the utility of ASAS. 

 1 2 3 4 5 6 9 

 

(39) The ability to maintain continuity of operations during displacement is a required 

capability. 

 1 2 3 4 5 6 9 

 

(40) The basic load of ASAS-unique digital maps is sufficient to support contingency 

operations. 

 1 2 3 4 5 6 9 

 

(41) Dedicated contractor ADP and hardware support is required to support the ASAS at 

an acceptable operational level. 

 1 2 3 4 5 6 9 

 

(42) The ability to exchange databases with the All Source enclave and the CE via the 

EDC message was effective. 

 1 2 3 4 5 6 9 

 

(43) ASAS does not present any Safety problems. 1 2 3 4 5 6 9 

 

(44) ASAS does not present any Health Hazards. 1 2 3 4 5 6 9 

 

(45) I was able to accomplish my individual critical tasks using ASAS while operating in 

MOPP IV. 

 1 2 3 4 5 6 9 

 

(46) I was able to accomplish my collective critical tasks using ASAS while operating in 

MOPP IV. 

 1 2 3 4 5 6 9 

(47) Additional personnel are not required to establish and sustain the operations of a 

field SCIF. 

 1 2 3 4 5 6 9 

 

(48) ASAS allows me to edit compartmented information prior to release to “protect” 

from unauthorized disclosure. 

 1 2 3 4 5 6 9 

 

(49) ASAS conforms with TEMPEST requirements and no TEMPEST violations were 

noted during ASAS operations. 

 1 2 3 4 5 6 9 

 

(50) The operator/analyst can easily use the ASAS to properly sanitize messages prior to 

dissemination to a collateral source. 

 1 2 3 4 5 6 9 

 

(51) The Single Source TEXTA database was complete and accurate and a useful 

COMINT tool. 

 1 2 3 4 5 6 9 

 

(52) The Single Source EPL database was complete and accurate and a useful ELINT 

tool. 

 1 2 3 4 5 6 9 

 

(53) The ASAS automated graphics tools (i.e., building AOIs, boundaries, etc.) were easy 

to use. 

 1 2 3 4 5 6 9 
78

FINAL DEBRIEF

ASAS SINGLE SOURCE DATA COLLECTOR QUESTIONNAIRE 

Purpose: This questionnaire is used to capture your views concerning the operational aspects and utility of ASAS. 

The data you provide will help decision makers evaluate how effective the ASAS has been designed to assist you in 

the performance of your intelligence gathering, processing and analysis tasks. 

PRIVACY ACT STATEMENT 

 Public Law 93-573, called the Privacy Act of 1974, requires that you be informed of the purpose and uses to 

be made of the information that is collected. 

 The Department of the Army may collect the information requested under authority of 10 United States Code 

137. 

 Providing information in this questionnaire is voluntary. Failure to respond to any particular question will 

not result in any penalty for the respondent. 

 The information collected in this survey will be used to evaluate military intelligence systems and their 

utilization. 

 The information will be used for research and analysis only. The US Army Operational Evaluation 

Command has primary research, analysis, and evaluation responsibility. 

Instructions: First, read the descriptions of the seven possible responses and then read each question carefully and 

circle the response that appropriately reflects your opinion. Although not required, we would welcome any written 

comment you may provide in response to a particular question. Please use the reverse of the page for these entries 

and key your responses to the question number. 

GENERAL INFORMATION. 

Name/Rank:_________________________ Position Observed: __________________ 
79

Rating Scheme: The rating criteria used for the following questions are defined below. Please ensure that you 

understand these rating criteria when answering questions with a rating scheme. 

1: Completely Disagree: There must be absolutely no doubt when using this response that the item being 

evaluated is of unacceptable design, composition, or value and must be completely redesigned, rewritten, or 

modified to be acceptable. 

2: Strongly Disagree: This response indicates that the item being evaluated is unacceptable and major 

improvements are required to make it acceptable. 

3: Generally Disagree: This response indicates the item being evaluated is unacceptable, but only minor 

improvements are required before it is acceptable. 

4: Generally Agree: This response indicates the item being evaluated is acceptable and helpful to the 

analyst/operator. 

5: Strongly Agree: This response indicates that the item being evaluated is very good and very helpful to the 

analyst/operator. 

6: Completely Agree: There must be absolutely no doubt when using this response that the item being evaluated 

cannot be any better designed, written, or has great value and is a desirable system feature or function. 

9: Unknown: This response indicates that you do not have sufficient knowledge to answer the question or you 

have not experienced the action or item in question. 
1 = Completely Disagree 3 = Generally Disagree 5 = Strongly Agree 9 = Unknown 

2 = Strongly Disagree 4 = Generally Agree 6 = Completely Agree

80

SINGLE SOURCE DATA COLLECTOR QUESTIONNAIRE 

(1) I am confident in my ability to properly populate, store, and maintain Single Source 

databases. 

 1 2 3 4 5 6 9 

 

(2) I am confident in the accuracy of the information contained in the Single Source 

databases. 

 1 2 3 4 5 6 9 

 

(3) ASAS significantly helped the analyst observed do his/her job as a COMINT 

Analyst. 

 1 2 3 4 5 6 9 

 

(4) ASAS assisted the analyst observed in producing a COMINT picture of the 

battlefield that was complete. 

 1 2 3 4 5 6 9 

 

(5) ASAS assisted the analyst observed in producing a COMINT picture of the 

battlefield that was accurate. 

 1 2 3 4 5 6 9 

 

(6) ASAS assisted the analyst observed in producing a COMINT picture of the 

battlefield that was timely. 

 1 2 3 4 5 6 9 

 

(7) ASAS significantly helped the analyst observed do his/her job as an ELINT Analyst. 1 2 3 4 5 6 9 

 

(8) ASAS assisted the analyst observed in producing an ELINT picture of the battlefield 

that was complete. 

 1 2 3 4 5 6 9 

 

(9) ASAS assisted the analyst observed in producing an ELINT picture of the battlefield 

that was accurate. 

 1 2 3 4 5 6 9 

 

(10) ASAS assisted the analyst observed in producing an ELINT picture of the 

battlefield that was timely. 

 1 2 3 4 5 6 9 

 

(11) The Target Analyst was able to significantly contribute to the ACE targeting 

process by identifying high payoff targets. 

 1 2 3 4 5 6 9 

 

(12) The time to develop, run, and display queries against ASAS databases does not 

adversely effect analysis. 

 1 2 3 4 5 6 9 

 

(13) The alarm process worked well. 1 2 3 4 5 6 9 

 

(14) Single Source graphics tools enabled the analyst to display and evaluate the effects 

that terrain and mobility factors may have on enemy courses of action. 

 1 2 3 4 5 6 9 

 

(15) The ability to automatically generate TACREPS/TACELINTS through the Single 

Source TACREP generator was effective. 

 1 2 3 4 5 6 9 

 

(16) The Single Source leadership is fully aware of the capabilities of ASAS. 1 2 3 4 5 6 9 

 

(17) The Single Source leadership is aware of the required interaction with the all source 

leadership to support ACE operations. 

 1 2 3 4 5 6 9 

 
1 = Completely Disagree 3 = Generally Disagree 5 = Strongly Agree 9 = Unknown 

2 = Strongly Disagree 4 = Generally Agree 6 = Completely Agree

81

(18) Based on my observations, the interactions with other Single Source Analysts 

enabled them to effectively contribute to answering PIRs. 

 1 2 3 4 5 6 9 

 

(19) ADP operations personnel appeared to be always able to identify the source of 

software problem. 

 1 2 3 4 5 6 9 

 

(20) ADP operations personnel appeared to be always able to access the command line 

and immediately fix software problems. 

 1 2 3 4 5 6 9 

 

(21) Single Source systems administrator requires access to system software through the 

command line. 

 1 2 3 4 5 6 9 

 

(22) ASAS operations would be significantly improved if a designated operator/analyst 

was trained in UNIX and had command line access. 

 1 2 3 4 5 6 9 

 

(23) ASAS enabled the analyst observed to handle the volume of messages received 

during the LUT. 

 1 2 3 4 5 6 9 

 

(24) The inability to maintain continual communication with external systems impacted 

on the ability to conduct Single Source analysis. 

 1 2 3 4 5 6 9 

 

(25) The Single Source enclave was easily displaced in MOPP 0. 1 2 3 4 5 6 9 

 

(26) The Single Source enclave was easily emplaced in MOPP 0. 1 2 3 4 5 6 9 

 

(27) The Single Source enclave was easily displaced in MOPP IV. 1 2 3 4 5 6 9 

 

(28) The Single Source enclave was easily emplaced in MOPP IV. 1 2 3 4 5 6 9 

 

(29) The analyst observed was able to accomplish analytical tasks using ASAS while 

operating in MOPP IV. 

 1 2 3 4 5 6 9 

 

(30) Software failures during the tactical phase of the LUT did not cause a loss of 

confidence in the system. 

 1 2 3 4 5 6 9 

 

(31) The majority of the software problems can be traced to database permissions being 

improperly set. 

 1 2 3 4 5 6 9 

 

(32) When a hardware problem occurred, military maintenance personnel responded in a 

timely manner. 

 1 2 3 4 5 6 9 

 

(33) When a hardware problem occurred, military maintenance personnel isolated the 

problem quickly. 

 1 2 3 4 5 6 9 

 

(34) When a hardware problem occurred, military maintenance personnel were able to 

fix the problem without difficulty. 

 1 2 3 4 5 6 9 

 

(35) ASL and PLL allocations at organic level appear to be sufficient. Downtime due to 

lack of spare parts was not experienced. 

 1 2 3 4 5 6 9 

 

(36) Operating under ACE Concepts and Doctrine enhances the ability to provide 

complete, timely, and accurate intelligence for tactical decision making by the G3 or 

operational staff.

 1 2 3 4 5 6 9 
1 = Completely Disagree 3 = Generally Disagree 5 = Strongly Agree 9 = Unknown 

2 = Strongly Disagree 4 = Generally Agree 6 = Completely Agree

82

 (37) Sufficient personnel were available to operate ASAS on a routine basis. 1 2 3 4 5 6 9 

 

(38) The ability to pass graphic formatted information to outside the ACE is critical to 

the utility of ASAS. 

 1 2 3 4 5 6 9 

 

(39) The ability to maintain continuity of operations during displacement is a required 

capability. 

 1 2 3 4 5 6 9 

 

(40) The basic load of ASAS-unique digital maps is sufficient to support contingency 

operations. 

 1 2 3 4 5 6 9 

 

(41) Dedicated contractor ADP and hardware support is required to support the ASAS at 

an acceptable operational level. 

 1 2 3 4 5 6 9 

 

(43) ASAS does not present any Safety problems. 1 2 3 4 5 6 9 

 

(44) ASAS does not present any Health Hazards. 1 2 3 4 5 6 9 

 

(45) The Operator/Analyst was able to accomplish the individual critical tasks using 

ASAS while operating in MOPP IV. 

 1 2 3 4 5 6 9 

 

(46) The Operator/Analyst was able to accomplish the collective critical tasks using 

ASAS while operating in MOPP IV. 

 1 2 3 4 5 6 9 

 

(47) Additional personnel are not required to establish and sustain the operations of a 

field SCIF. 

 1 2 3 4 5 6 9 

 

(48) ASAS allows the Operator/Analyst to edit compartmented information prior to 

release to “protect” from unauthorized disclosure. 

 1 2 3 4 5 6 9 

 

(49) ASAS conforms with TEMPEST requirements and no TEMPEST violations were 

noted during ASAS operations. 

 1 2 3 4 5 6 9 

 

(50) The Operator/Analyst can easily use the ASAS to properly sanitize messages prior to 

dissemination to a collateral source. 

 1 2 3 4 5 6 9 

 

(51) The Single Source TEXTA database was complete and accurate and a useful 

COMINT tool 

 1 2 3 4 5 6 9 

 

(53) The ASAS automated graphics tools (i.e., building AOIs, boundaries, etc.) were easy 

to use. 

 1 2 3 4 5 6 9 
83

2.3.2 Use of Model 2 for MANPRINT Evaluation of the Heavy Equipment Transporter 

System (HETS) 

The model designated as model 2 appears to work well with soldiers who have relatively low 

verbal skill like those who operate and maintain the Heavy Equipment Transporter System 

(HETS) and when the critical tasks have been identified. 

2.3.2.1 Resources Used for MANPRINT Evaluation of HETS. In this test, there were about 

50 players who were responsible for operating and maintaining the system. In order to collect 

sufficient Reliability Availability and Maintainability (RAM) data on the HETS equipment, the 

test took 6 months to complete. The MANPRINT Manager developed the data collection plan, 

designed the data collection instruments and directed the data collection process with two 

MANPRINT support personnel. 

2.3.2.1.1 Acquiring a Contractor to Support MANPRINT Evaluation. When assigned to the 

Heavy Equipment Transporter System (HETS) test as MANPRINT Manager, the Test Officer 

indicated that one of their contract data collectors could be dedicated to support the MANPRINT 

effort. The one selected had system knowledge—he knew automotive equipment, had rebuilt 

vehicles, and had data processing skills. He had been a U.S. Army Aviation Warrant Officer 

who was riffed out of the service. Following selection of this individual to support the 

MANPRINT effort, the data collection and analysis effort was scoped out in greater detail. 

2.3.2.1.2 Acquiring a TEMP-Hire to Support MANPRINT Evaluation. As the data collection 

process was progressing, Operational Evaluation Command (OEC), now Army Evaluation 

Command (AEC), indicated that they didn't think there was enough support for the MANPRINT 

effort; accordingly the MANPRINT effort was supplemented with support from a government 

TEMP-hire worker. As the MANPRINT data collection process was already under way, it 

seemed appropriate to begin by letting the TEMP-hire work with the original contractor data 

collector to get him “read-in.” Conceptually, the plan seemed appropriate, but being new to 

supervisory responsibilities, it was not understood that contract personnel cannot direct/supervise 

government employees. Early-on the arrangement seemed workable, but the contract data 

collector began to be more directive rather than advisory when the MANPRINT Manager was 

not present. When the TEMP hire documented this conflict in writing and pointed out the 

inappropriate supervisory role that had emerged, it was again necessary to re-scope the effort. 

2.3.2.2 Data Collection Procedures for MANPRINT Evaluation of the HETS. The basic data 

collection strategy for this effort involved conducting a series of individual interviews with each 

of the system operators (drivers and assistant drivers) and system maintainers. 

2.3.2.2.1 MANPRINT Interview Protocol Used With HETS Operators and Maintainers. The 

interviews involved use of a common set of questions comprised of about four pages for each of 

the operational and maintenance critical tasks keyed to each of the MANPRINT domains 
84

(see page 85). The set of interview questions for each of the operator critical tasks were bound 

together in a booklet; a comparable booklet was compiled for system maintainers. 

2.3.2.2.2 Emerging Concerns for a HETS Task. To support the interview process, a booklet 

containing sheets documenting the emerging concerns associated with performing each task was 

provided for reference and as a possible memory jogger for each interviewee (see page 89). As 

each interview was conducted, new concerns addressed by the interviewee were added to this 

list. 

2.3.2.2.3 Technical Manual Procedure for Performing Task Being Interviewed. During 

discussion of the interview procedure with the Training Systems Manager (TSM), he expressed 

concern that the interviews were being conducted within a shelter rather than outside where the 

M1070 cab and M1000 semi-trailer were located. His concern was that when a problem is noted, 

ambiguity about the equipment being discussed could arise. To address this concern and to 

provide a comfortable environment for the soldiers to formulate their thoughts, the interview 

process was facilitated by use of tabbed pages from the appropriate Technical Manuals (TM) that 

showed step by step procedures (with illustrations) for conducting the task currently under 

discussion (table 17, see page 90). During the interview process, the interviewer usually sat 

opposite the soldier and the booklets containing the emerging list of concerns and the TM could 

be easily rotated on the table so that the interviewee could be very specific about his problem 

descriptions. 

2.3.2.2.4 Supplementary MANPRINT Data Collection. Conflict between Contract and TEMPHire cited in section 2.3.2.1.2, in part, stimulated the need to re-scope the MANPRINT 

Evaluation effort. Emerging findings from early interviews had indicated that there were 

concerns about temperature and noise in and around the HETS cab. As the first set of interviews 

had been pretty much completed by this time, the TEMP hire was directed to get some hard data 

to determine whether there really was a temperature and noise problem. In order to collect this 

data, the TEMP hire was provided with a sound pressure meter and thermometer and instructed 

to make arrangements to go out with the drivers. He was given an experimental design for 

collecting temperature and noise data. Results of that data collection effort are presented in 

appendix I. Studying the results tables and the accompanying figure should provide sufficient 

information for discerning that design. Presentation of this description indicates the multiplier 

effect of providing additional support to the MANPRINT data collection effort. Had the 

contractor and TEMP support not been provided, the MANPRINT manager would probably have 

had to be content with completing a single set of interviews (conducted by himself) involving 

each HETS operator and maintainer. 
85

HETS MODEL 2 USED WITH LIST OF CONCERNS AND TABBED TMS 

17. TASK: Load a Disabled Payload on a M1070 Tractor and a M1000 Semi-trailer Combination Using Dual 

Winches (-14 pp. 2-125 to 2-130) 

1. Did you perform or observe this task being performed? (Y N ) 

 If No, go to sheet for next TASK. 

2. Problems? 

 a. Have you had any problems in performing this task? (Y N ) 

 1) If No, go to 2b. 

 2) If Yes, please describe: _______________________________________ 

_____________________________________________________________________________ 

 b. Are you aware of problems others have had in performing this task? 

 (Y N ) [refer to sheet of concerns] 

 1) If No to 2a and 2b, to 6. 

 2) If Yes, please describe: _______________________________________ 

_____________________________________________________________________________ 

3. Which concerns are most important (up to 3)? _____________________________ 

_____________________________________________________________________________ 

Indicate if there are other problems which you do not see on the sheet of concerns: 

____________________________________________________________________ 

_____________________________________________________________________________ 

Problem Frequency (when ~ task is being performed) Circle letter indicating judgment. 

 CONCERN

 1st 2nd 3rd 

 A A A Frequent -- continuously experienced 

 B B B Probable -- will occur several times 

 C C C Occasional -- will occur several times 

 D D D Remote -- unlikely but possible 

 E E E Improbable -- very unlikely to occur 

Problem Severity: Circle Roman numeral indicating judgment. 

CONCERN

1st 2nd 3rd 

 I I I Catastrophic -- death or systems loss 

 II II II Critical -- severe injury or major system damage 

 III III III Marginal -- minor injury or minor system damage 

 IV IV IV Negligible -- less than minor injury or system damage 
86

4. If Yes to 2, could one or more of these problems have been avoided by using: 

 a. More people? ( Y N ) 

 1) If Yes, how many more? 

 

 b. A different procedure? ( Y N ) If Yes, please describe: 

 1) What is wrong with current procedure? 

____________________________________________________________________________________ 

____________________________________________________________________________________ 

 2) What changes can you recommend? 

____________________________________________________________________________________ 

____________________________________________________________________________________ 

 3) What modification of the procedure might make the task “doable” with only two soldiers? 

____________________________________________________________________________________ 

____________________________________________________________________________________ 

 c. An equipment modification ( Y N ) 

 1) Please describe your recommendations: 

____________________________________________________________________________________ 

____________________________________________________________________________________ 

 

 d. More lecture training? ( Y N ) 

 e. More hands-on training? ( Y N ) 

 f. Another training procedure? ( Y N ) 

 1) If Yes, please describe. 

____________________________________________________________________________________ 

____________________________________________________________________________________ 

 g. Greater caution? ( Y N ) 

 1) If Yes, does performing this task create a safety hazard (for the soldier or equipment)? 

 ( Y N ) 

 2) If Yes, please describe. 

____________________________________________________________________________________ 

____________________________________________________________________________________ 
87

5. Job Stress 

 a. Please indicate on the following scale the extent this task must be performed at the same (or nearly the 

same) time as other tasks (TIME STRESS) 

 1 = No overlap 

 2 = Little overlap 

 3 = Occasional overlap 

 4 = Frequent overlap 

 5 = Very frequent overlap 

If 4 or 5, which other task(s)? _ 

 b. Please indicate on the following scale the extent conscious mental effort (or concentration) is required 

to perform the task (MENTAL STRESS) 

 

 1 = None needed 

 2 = Little needed 

 3 = Moderate amount needed 

 (a) Why?_______________________________________________________ 

_____________________________________________________________________________________________

___________________________________________________________________________ 

 4 = Extensive amount needed 

 (a) Why?______________________________________________________ 

___________________________________________________________________________________ 

 c. Please indicate on the following scale the extent performing this task requires physical effort

(PHYSICAL STRESS). 

 1 = No demand 

 2 = Little demand 

 3 = Moderate demand 

 4 = Heavy demand 

 5 = Very heavy demand 

 (a) Why?_________________________________________________________ 

_____________________________________________________________________________________________ 

 d. Please indicate on the following scale the extent performing this task causes (for Operators: you to 

feel) (for Data Collectors: you to observe signs of) confusion, frustration, or anxiety (PSYCHOLOGICAL STRESS) 

 1 = None 

 2 = Little 

 3 = Moderate 

 4 = High 

 a) Why?__________________________________________________________ 

_______________________________________________________________________________ 

 5 = Intense 

 (a)Why?______________________________________________________________________ 

_________________________________________________ 
88

6. Manual Description of Procedure for Performing This Task (turn to pages in TM for this task) 

(Y = Acceptable N = Changes Needed) 

 a. Are you aware of anything which is not technically correct in the: 

 1) Written procedure ( Y N ) 

 (a) If Yes, describe problem. 

____________________________________________________________________________________________ 

____________________________________________________________________________________________ 

____________________________________________________________________________________________ 

____________________________________________________________________________________________ 

 2) Figures or drawings ( Y N ) 

 (a) If Yes, describe problem. 

____________________________________________________________________________________________ 

____________________________________________________________________________________________ 

____________________________________________________________________________________________ 

____________________________________________________________________________________________ 

 b. Are you aware of anything which is not understandable in the:_______________________________ 

 1) Written procedure ( Y N ) 

 (a) If Yes, 

why?________________________________________________________________________________________ 

____________________________________________________________________________________________ 

 2) Figures or drawings ( Y N ) 

 (a) If Yes, 

why?________________________________________________________________________________________ 

____________________________________________________________________________________________ 
89

CONCERNS ASSOCIATED WITH

17. TASK: Load a disabled payload on a M1070 tractor and a M1000 

 semi-trailer combination using dual winches 

9. (S) WINCH OPERATION — Can't see cables, must use spotter. 

10. (S) ENGINE KILL SWITCH — Required, PTO is in CAB, too far. 

15. PTO SWITCH/LIGHT — Spacing too far, relocate. 

21. (S) CHAINING A TANK — Must reach, arms extended, with chains (30—100 lbs) can present potential 

back injury situation. Females only 65% (average) of males upper body strength. Must crawl under load to rig it. 

23. PAYLOAD CHOCKS — Replace nuts and bolts with quick release pins. 

29. (S) LOADING, WINCHING, SECURING LOAD — Load old system in 15 minutes, this one takes an 

hour. Can't see winch cables, tie downs are not user friendly, must climb under load to secure it, must count chain 

links. 

31. TIE DOWNS — Not enough, relocate, strength, no. of shackles.
90

Figure 1. Technical manual procedure for performing task being interviewed. 
91

Figure 1. Technical manual procedure for performing task being interviewed (continued). 
92

Figure 1. Technical manual procedure for performing task being interviewed (continued). 
93

Figure 1. Technical manual procedure for performing task being interviewed (continued). 
94

Figure 1. Technical manual procedure for performing task being interviewed (continued). 
95

Figure 1. Technical manual procedure for performing task being interviewed (continued). 
96

2.3.3 Use of Model 2 for MANPRINT Evaluation of the Enhanced Position Location 

Reporting System (EPLRS)

Model 2 was also employed for MANPRINT Evaluation of the Enhanced Position Location 

Reporting System (EPLRS) IOTE (see footnote c, table 3, page 11). The model appeared to work 

reasonably well with General Purpose User (GPU) soldiers with varying levels of verbal skill. 

2.3.3.1 Resources Used for MANPRINT Evaluation of EPLRS. Players in this test included 

about 200 soldiers with a variety of MOSs and skill levels. The test was conducted in three 

phases during a period of one month. MANPRINT support was provided by a GS-12 Manager 

and three GS 5 TEMPS—one of whom was experienced in supporting MANPRINT evaluations 

for other systems under tests. She served as the “in the field” data manager who provided day to 

day direction to other TEMPS and guidance to four military data collectors. The MANPRINT 

Manager developed the Data Collection Plan, designed the data collection instruments and 

coordinated with Test Management Team to assure that agreed upon arrangement for data 

collection remained in place. The GS 5s and supporting military collected and processed the 

data and provided the raw data and frequency tabulations to the MANPRINT Manager. 

2.3.3.2 Data Collection Procedures for MANPRINT Evaluation of EPLRS. In contrast to the 

HETS test where there were only about 50 soldier participants and a long test window 

(6 months), the EPLRS test involved about 200 soldier participants for a 30-day test. 

Consequently interviews had to be more targeted to those soldiers who had problems on specific 

tasks. All participating soldiers completed one of seven forms like that shown on page 97—

containing different sets of tasks for the different operator types. In each case they checked 

those critical tasks on which they experienced one or more problems—and for which more 

detailed MANPRINT data appeared necessary. The second part of this data collection effort (see 

page 98) involved use of an a MANPRINT Evaluation form that contained interview questions 

paralleling those used for the HETS operators' and maintainers' interviews (see page 15). 

Appendix J provides a slightly different version of the MANPRINT Biographical Questionnaire 

used for this system evaluation that includes additional biographical information and addresses 

potentially important social attitudinal factors. 
97

Part 1 

MANPRINT DEBRIEF FOR NCS OPERATORS 

BIO/BACKGROUND/POSITION

Name ________________________________ PIN No. ___________ Date ________________ 

 Mo/Da/Yr 

Please circle:

Test Phase: Pilot Stand-alone Integrated 

MOPP: 0 1 2 3 4 

INSTRUCTIONS

RESPOND FOR EACH TASK: 

1. Circle Y in PROBLEM column if completion of the task created a performance problem for you since you last 

completed this form. 

IF TEST PHASE IS PILOT, CIRCLE Y FOR PROBLEMS WHICH OCCURRED ONLY AFTER PILOT PHASE 

BEGAN 

2. For those cases where you circled Y in the PROBLEM column, if you believe your problem was caused by a 

training deficiency, circle Y in the TRAINING column. 

Task No. Task Name Problem 

Training

Related 

1 List History, Need, Purpose and Key Concepts of EPLRS Y Y 

2 List Major EPLRS Assemblies and Match Purpose of Each Y Y 

3 List NCS Controls and Indicators and Match Purpose of Each Y Y 

4 List EPLRS Safety, ESD and Security Requirement Y Y 

5 Perform System Power-On Procedure Y Y 

6 Perform Key Load Operations Y Y 

7 Perform Program Load Procedures Y Y 

8 Perform Initialization Procedures Y Y 

9 Perform System Power-Down Procedure Y Y 

10 Enter, Update, Delete and Record Library Data Y Y 

11 Enter, Update, Delete and Record Map Data Y Y 

12 Perform Basic Operator Control Switch Actions Y Y 

13 Perform Map Tape Building/Updating Switch Actions Y Y 

14 Perform Display Manipulation Switch Actions Y Y 

15 Perform Map Scale/Offset Switch Actions Y Y 

16 Enter System Configuration Parameters Y Y 

17 Resolve Alerts Advisory Y Y 
98

Part 2 

MANPRINT ASSESSMENT 

BIO/BACKGROUND/POSITION

Name ________________________________ PIN No. _________ Date ___________ Time _________ 

 Mo/Da/Yr 0000-2400 

Please circle:

Position: NCS Oper RS Oper NCS Maint RS Maint EGRU/Gateway Oper ETS DS Maint SYSCON 

Test Phase: ETE Pilot Stand-alone Integrated 

MOPP: 0 1 2 3 4 

 

PROBLEM DESCRIPTION

1. Task Performing (use task no. from Critical Task List when possible): 

____________________________________________________________________________________________ 

____________________________________________________________________________________________ 

2. Problem Description: 

____________________________________________________________________________________________ 

____________________________________________________________________________________________ 

3. Most probable problem cause (circle one): 

 a. Equipment malfunction (appears unrelated to any soldier action) 

 b. Training 

 1) Poor or inadequate classroom training on task 

 2) Lack of sustainment/collective training on task 

 c. Manpower (not enough soldiers to do job involving this task) 

 d. Personnel (task is outside “normal” duties for my MOS or skill level) 

 e. Health Hazard (task jeopardized my or another crew member's safety) 

 f. System Safety (task performed created a safety problem related to EPLRS equipment) 
99

 g. Human Factors 

 1) Problem working with EPLRS equipment/hardware--difficult or complicated 

 to work with or access when this task was performed 

 2) Problem with EPLRS software when this task was performed 

4. Problem reported previously: YES NO (circle) 

 If YES, go to item 9; otherwise continue with item 5. 

5. Frequency this task was performed since class instruction--BEFORE PILOT TEST BEGAN (use number from 

following scale) ____________ 

1 = at least once daily 

 2 = once or twice a week 

 3 = at least every two weeks 

4 = at least once a month 

 5 = not at all 

6. TM error contributed to this problem: YES NO (circle) 

 If YES indicate: a) TMno. b) page/para or Figure c) error description 

 a) TM No. ______________ 

 b) _______________________________________________________________________________________ 

 c) ________________________________________________________________________________________ 

____________________________________________________________________________________________ 

7. Problem Frequency/Severity (when task is being performed): 

Frequency (circle letter indicating judgment) 

a. Frequent - continuously experienced 

b. Probable - will occur frequently 

c. Occasional - will occur several times 

d. Remote - unlikely, but possible 

e. Improbable - very unlikely to occur 

Severity (circle Roman numeral indicating judgment) 

I Catastrophic - death or system's loss 

II Critical - severe injury or major system damage 

III Marginal - minor injury or system damage 

IV Negligible - less than minor injury or system 

 damage 
100

8. Job Stress (when task is being performed): 

a) TIME STRESS - Task must be performed at or 

nearly at same time as other tasks. 

Circle No.

1 = No overlap 

2 = Little overlap 

3 = Occasional overlap 

4 = Frequent overlap 

5 = Very frequent overlap 

If 4 or 5, which other tasks (use nos. from Critical 

Task List, if possible) _______ _______ _______ 

b) MENTAL STRESS - Task completion requires 

conscious mental effort (concentration) 

Circle No.

1 = None needed 

2 = Little needed 

3 = Moderate amount needed 

4 = Extensive amount needed 

If 3 or 4, indicate why: 

________________________________________ 

________________________________________ 

c) PHYSICAL STRESS - Task completion requires 

physical effort. 

Circle No.

1 = No demand 

2 = Little demand 

3 = Moderate demand 

4 = Heavy demand 

5 = Very heavy demand 

If 4 or 5, indicate why: 

________________________________________ 

d) PSYCHOLOGICAL STRESS - Task completion 

causes confusion, frustration, or anxiety.

Circle No.

1 = None 

2 = Little 

3 = Moderate 

4 = High 

5 = Intense 

If 4 or 5, indicate why: 

________________________________________ 

9. Mission successfully performed (even with problem): YES NO (circle) 

10. Fix (procedure/task modification needed to complete mission): YES NOT (circle) 

11. Recommended fix (if any) (continue on back side, if necessary): _____________________________ 

____________________________________________________________________________________ 

 __________________________ ___________ 

 On-Site Data Collector PIN 

 __________________________ ___________ 

 MANPRINT Data Collector PIN 
101

2.3.4 Use of Model 3 for MANPRINT Evaluation of ASAS in the War Fighting Rapid 

Acquisition Program (WRAP) 

This evaluation involved ten different types of ASAS functions with no well defined critical task 

lists that could be compiled within the short response time available. Whereas some tests are 

designed primarily to provide opportunity for evaluation—with any training function being 

afforded as secondary in purpose—this effort was designed primarily as a training exercise with 

an evaluation function being superimposed (and secondary) on the behaviors exhibited during 

that training. 

2.3.4.1 Resources Used for MANPRINT Evaluation of ASAS during a WRAP. This training 

exercise was conducted as a three day activity in which only one data collection resource was 

available to provide MANPRINT Evaluation for soldiers performing in the ASAS functionality. 

Contributing to the uniqueness of the methodology employed in this evaluation was its use to 

obtain quality MANPRINT data when the effort required a quick turn-around activity. Support 

for this assessment began on Veteran's Day in 1997. One of the Fort Hood Field Element 

personnel was tasked to coordinate the evaluation effort that involved several MANPRINT 

Analysts, but generally only one per Battle Field Functional Area (BFA). Some of these 

Analysts came from U.S. Army Research Laboratory Headquarters and some from other Field 

Elements. The author was contacted at about 10 a.m. and asked if he could support this effort to 

collect MANPRINT data for ASAS. With about three hours to plan how this support could be 

provided, it was very relevant to ask what kind of meaningful and potentially useful 

MANPRINT Evaluation could be accomplished with so little time for preparation. 

2.3.4.2 Data Collection Procedures for MANPRINT Evaluation of ASAS during a WRAP. An 

interview protocol containing generic MANPRINT questions (see table 8) was the basic data 

collection instrument used during the WRAP for ASAS. With the soldier sitting alongside the 

MANPRINT Analyst, each MANPRINT question was read to the soldier, first, to determine 

whether there was any likely MANPRINT problem. For those questions where possible 

problems did exist, the Analyst asked for description of the problem(s) and continued with 

follow-up questions until a relatively complete understanding of the problem existed. Soldiers 

were then asked for their recommendations on how the problem(s) could be corrected and the 

consequences likely if the problem(s) was (were not) fixed. Data analysis involved tabulating 

reported problems by MANPRINT domain, the recommended procedures for correction and the 

consequences of not fixing the problem. This instrument has proven to be a useful quick 

evaluation instrument on several occasions for collecting MANPRINT data and especially useful 

when you don't know very much about the system. For example, when table 8 is used to support 

MANPRINT Evaluation for the SMART-T in the FOTE, generally “SMART-T” was substituted 

for ASAS throughout. 
102

Table 8. MANPRINT interview questions used during WRAP for ASAS. 

During Your Mission, Did You ... (Repeat For Each Item Below): N Y 

MANPRINT Domain 

Addressed 

1. See any task or operation which jeopardized your safety or 

 that of any other crewmember? 

N Y Health Hazard 

2. Note any safety problem (actual or potential) that would 

degrade (or damage) equipment, configurations, procedures or 

 work conditions? 

N Y System Safety 

3. Experience or suspect any condition associated with 

Operating the ASAS which you consider unhealthy? 

N Y Health Hazards 

4. Or any other crew member receive any injury related to the 

ASAS operations? 

N Y Health Hazard 

5. Need to perform any task or operation for which you were 

not trained? 

N Y Training 

6. Perform any task or operation for which your training was poor or 

inadequate? 

N Y Training 

7. Note any crew/collective task degraded because instruction 

was poor or tasks taught incompletely? 

N Y Training 

8. Feel that your or your crew's performance was degraded or 

“held-down” by any training shortfall? 

N Y Training 

9. Perform any task or operation that in your judgment 

required an additional crew member (or more time than 

allowed for you to do alone)? 

N Y Manpower 

10. Need to work harder with ASAS to augment MI assessment than 

with manual MI procedures? 

N Y Manpower 

11. Perform any task or operation that was outside of “normal” duties for 

your MOS and skill level? 

N Y Personnel 

12. Note any ASAS equipment/hardware that was unnecessarily difficult, 

complicated, or “unfriendly” to work with? 

N Y Human Factors 

Engineering: Hardware 

13. Experience any problem associated with ASAS software at your crew 

station? 

N Y Human Factors 

Engineering: Software 

14. Identify any ASAS equipment/hardware placement which kept you 

or your crew from “maxing out”? 

N Y Human Factors 

Engineering: Hardware 

15. Note any ASAS equipment/hardware placement which made access 

or use difficult? 

N Y Human Factors 

Engineering: Hardware 

16. Note any ASAS-related procedures which were unnecessarily 

difficult, complicated, or “unfriendly”? 

N Y Human Factors 

Engineering: Procedures 

17. Identify any “dumb” things the ASAS required you to do? N Y Human Factors 

Engineering: Procedures 
103

2.3.5 Use of Model 4 for MANPRINT Evaluation of the Secure Mobile Anti-Jam Reliable 

Tactical Terminal (SMART-T) in the FOTE 

In another case, it was necessary for a second MANPRINT Analyst to replace the assigned 

MANPRINT Analyst after the Pilot Test for a Follow-on Operational Tests and Evaluation 

(FOTE) of the SMART-T. The first time that this system was observed by the second Analyst 

was following the Pilot Test of the FOTE as it was deployed in the field. 

2.3.5.1 Resources Used for MANPRINT Evaluation of the SMART-T in the FOTE. In this test, 

there were about 30 soldiers who participated as operators; system operation was conducted over 

a three day period, 24 hr per day. The MANPRINT Analyst reviewed and modified the data 

collection plans and instruments used planned for use in the MANPRINT evaluation; ~20 of the 

30 participating soldiers were interviewed with the instrument shown in table 8 and data were 

analyzed and a report of findings was prepared for the Test Officer (15). 

2.3.5.2 Data Collection Procedures for MANPRINT Evaluation of the SMART-T. The 

interview instrument (see table 8) was used in a manner very similar to its use in the ASAS 

MANPRINT Evaluation during the WRAP (see paragraph 2.3.4.2). Other group-oriented 

MANPRINT evaluation instruments were administered to test players by other test personal and 

results were provided to the MANPRINT Analyst for analysis and summarization. 

2.3.6 Use of Model 4 for MANPRINT Evaluation of the M1A2 in the IOTE 

Soldiers participating in this test were relatively “low” in verbal skills and a usable list of critical 

tasks did not appear available. 

2.3.6.1 Resources Used for MANPRINT Evaluation of the M1A2. In the M1A2 IOTE, there 

were about 80 armor MOSs. Supporting this effort were a GS13 (MGR) who developed the data 

collection plan, designed the data collection instruments and supervised the collection and 

processing of data; participating in the data collection and analysis were a senior retired NCO 

with system experience and four retired NCOs who also had system experience served as Temps. 

2.3.6.2 Data Collection Procedures for MANPRINT Evaluation of the M1A2. This data 

collection procedure included use of an MANPRINT Interview Schedule comparable to that used 

in Model 3 (table 8). In implementing this model, there was an attempt to validate the reported 

problem, to acquire a Risk Assessment for Health Hazards and System Safety (page 104) and 

preparation of a MANPRINT Problem/Suggestion Report (page 105). In completion of this 

latter form, the problems reported were characterized by Type (MANPRINT domains) and 

Mission Impact; data collection was designed to describe problems and included opportunity to 

acquire suggestions for remediation. 
104

SME/DATA COLLECTOR VALIDATION INSTRUCTIONS 

 Review the description of the MANPRINT problem. 

 Attempt to validate this problem though one of the following means. 

 Someone else who may have knowledge of problem 

 Interview the operator/RAM data collector/performance data collector who wrote the MPR 

 Review video tape of problem 

 After you have located one of the above sources and verified the problem, ask the following 

 questions and record answers on the MPR. 

 1. What were the contributing causes to the problem? 

 2. What are the likely consequences if the problem is not fixed? 

 3. What are the solutions to this problem? 

 Remember to complete the MPR by doing the following:

 Mark the space on the MPR showing how you validated the problem. 

 Be certain to fill in your evaluation of the Type and Impact of the problem. 

 Use the Safety Hazard and Hazard Probability scales to determine Safety Category and Probability codes 

at the bottom of the MANPRINT Problem/Suggestion Report. 

SAFETY HAZARD CATEGORY 

 Catastrophic I Death or system loss 

 Critical II Severe injury or major system damage 

 Marginal III Minor injury or system damage 

 Negligible IV Less than minor injury or system damage 

HAZARD PROBABILITY CATEGORIES 

Description Level Specific Individual Item Inventory

Frequent A Likely to occur frequently Continuously experienced 

Probable B Will occur several times in life of an item Will occur frequently 

Occasional C Likely to occur sometime in life of an item Will occur several times 

Remote D Unlikely, but possible to occur in life of item Unlikely, but can reasonably 

 be expected to occur 

Improbable E So unlikely, it can be assumed as not Unlikely to occur, but possible 

 occurring may not be experienced 
105

 MANPRINT Log No._______________ 

M1A2 BLOCK II IOTE: MANPRINT PROBLEM/SUGGESTION REPORT 

Operator: ________________________________ Rank: ____________ Date: ______________________ 

Data Collector: ______________________________________________ PIN No. ___________________ 

_ 

Bumper No.: ______________________ Vehicle SN: __________________ 

INSTRUCTIONS: As you work with the M1A2, you may notice or experience a problem or have a suggestion for 

improving the system. For any problem or incident you encounter, indicate the time it occurred and describe it 

clearly. Using the scales below, indicate the type of problem and its impact on the mission. Simply describe 

suggestions. To list more than one problem or suggestion, draw a line across the page after each complete entry. 

 

Type Mission Impact 

E = Equipment (hardware) 

C = Computer software 

T = Training (no/inadequate) 

S = Safety/Health Hazard*

W = Workload/Manpower 

O = Operator (mistake/memory) 

S = Severe (probable failure) 

Mo = Moderate (probably degrading) 

Mi = Minimal (mainly attitudinal) 

N = Negligible (essentially none) 

TIME DESCRIPTION TYPE IMPACT 

SHORT TITLE: 

 SME Badge No.__________ 

 

MPT Team O None O Observed O Op Interview O SME O Video 

VALIDATION O Invalid O Valid- Type Code ______ Impact Code _______ 

 * Safety Category _______ Probability ________ 

DAG Approval _______ DTG ______________________ 
106

3. Conclusions 

The purpose of this report has been to identify alternative procedures for conducting 

MANPRINT evaluations. After characterization of the areas (domains) addressed by 

MANPRINT, models for evaluation were discussed. For each of the models, procedures for 

implementing MANPRINT evaluations were illustrated by presentation of different data 

collection instruments. The appendices contain additional forms and some detailed procedures 

about how those data are used—specifically for the Personnel, Health Hazards, and System 

Safety domains. 
107

4. References 

1. TEXCOM Memo 73-1, Test Officers’ Planning Manual, Chapter 5. U.S. Army Test and 

Experimentation Command: Fort Hood, TX, October 1991. 

2. AR 602-2. Manpower and Personnel Integration (MANPRINT) in the System Acquisition 

Process 2001. 

3. Filep, R. T.; Heuckeroth, O. H.; Eastwood, G. B.; Sales, T. B. The Design of a Management 

System for the Conduct of Special Education Within Los Angeles County. Institute for 

Educational Development, El Segundo, CA, 1971, pp 17–18. 

4. DA PAM 611-21. Personnel Selection and Classification Military Occupational 

Classification and Structure. http://www.usapa.army.mil/pdffiles/(accessed March 1999). 

5. MIL-STD-1472D. Human Engineering Design Criteria for Military Systems, Equipment, 

and Facilities 1989. 

6. MIL-STD-1472F. Department of Defense Design Criteria Standard 1999. 

7. Babbitt, B. A.; Nystrom, C. O. Questionnaire Construction Manual; performed by Essex 

Corporation: Westlake Village, CA, for U.S. Army Research Institute for Behavioral and 

Social Sciences (ARI), June 1989, ARI Research Product 89-20. 

8. Heuckeroth, O. H. Evaluation of Selected MANPRINT Issues for Product Improvement 

Program of the OQ-290(V)1 Electronic Equipment Test Facility; WP FH 88-15; U.S. Army 

Research Institute for Behavioral and Social Sciences: Alexandria, VA, December 1988. 

9. Morrison, A. F.; Paffenroth, D. R. Regency Net (RN): Initial Operational Test and 

Evaluation; 900-OT-1079; U.S. Army Operational Test and Evaluation Command: 

Alexandria, VA, March 1991. 

10. MIL-STD-882A. System Safety Program Requirements 1977. 

11. TOP-1-1-012. Defense Technical Information Center, Defense Logistics Agency. U.S. 

Army Test and Evaluation Command Test Operations Procedures. Classification of 

Deficiencies and Short-Comings. U.S. Army Test and Evaluation Command: Aberdeen 

Proving Ground, MD, April 1979. (Document obsolete, essential content abstracted in 

appendix E.) 

12. Corporate Authorship (informal document prepared for Operational Evaluation Command). 

MANPRINT Assessments of Seven FY 98 Warfighting Rapid Acquisition Program (WRAP) 

Candidates. U.S. Army Research Laboratory: Aberdeen Proving Ground, MD, Human 

Research and Engineering Directorate, November 1997.
108

13. Gibson, J. L.; Davis, D. Heavy Equipment Transporter System (HETS) (Tractor, Trailer) 

Full Evaluate Test Report: Initial Operational Test; 91-OT-1229; U.S. Army Test and 

Experimentation Command: Fort Hood, TX, December 1993. 

14. Sledge, N. H., Jr.; Wilson, S. M.; Hanna, J. B.; York, D. W. Test and Evaluation Report 

(TER) for the M1A2 Abrams Tank System Initial Operational Test and Evaluation, 

Volume II. Appendices A Through R. 93-OT-1358A; U.S. Army Operational Test and 

Evaluation Command: Alexandria, VA, May 1994. 

15. Rahman, M.; Heuckeroth, O.; Borrero, E.; Andrulonis, R.; Gonzales, B. P. System 

Evaluation Report for the Secure, Mobile, Anti-Jam, Reliable, Tactical Terminal 

(SMART-T). Follow-on Test and Evaluation (FOT&E), ADB 275077 (DTIC), U.S. Army 

Test and Evaluation Command: Alexandria, VA, February 2002. 

16. Smith, R.; Kime, M. All Source Analysis System (ASAS) Block II Initial Operational Test; 

2005-OT-C4-ABCS1-A3219; U.S. Army Operational Test Command (USAOTC): Fort 

Hood, TX, April 2005. 

17. Sul, H. P.; Buche, C. J. Test and Evaluation Report for the Enhanced Position Location 

Reporting System (EPLRS), Initial Operational Test and Evaluation; 95-OT-1140A; U.S. 

Army Operational Test and Command: Alexandria, VA, February 1995. 

18. Nott, M. B. Maneuver Control System (MCS) Initial Operational Test and Evaluation; 

98-OT-1261D; U.S. Army Test and Experimentation Command: Fort Hood, TX, July 1998. 
109

Appendix A. Memorandum for Defense Manpower Data Center (DMDC) to 

Support Sample Representativeness Comparisons 

 

This appendix appears in its original form, without editorial change. 
110

AMSRL-HR-MV (70-1y) 14 May 1999 

MEMORANDUM FOR DEFENSE MANPOWER DATA CENTER (DMDC), DOD CENTER, 

 MONTEREY BAY, ATTN: , 400 GIGLING 

 ROAD, SEASIDE, CA 93955-6771 

SUBJECT: Request for ASVAB and Related Data 

1. Per recent FONECON with about expected SUBJECT request, paragraphs 

2-6 detail the current request being made. As this request is very similar to a recent effort completed by DMDC 

Analyst (insert name), you may wish to assign this current effort to him. 

2. For all soldiers on the enclosed floppy1

, please provide individual scores and group mean and standard 

deviation for each of the listed variables (except as noted): 

 Date of Birth—YR (no mean or standard deviation) 

 Date of Birth—MO (no mean or standard deviation) 

 Date of Birth—DAY (no mean or standard deviation) 

 Sex (Gender) (no mean or standard deviation) 

 Race Ethnic (no mean or standard deviation) 

 Paygrade (no mean or standard deviation) 

 Education (variable in column 11 on Active Duty Military Master and Loss 

 Edit file) 

 (no mean or standard deviation) 

 Highest Year Education (variable in column 24 of the Active Duty...file) 

 (no mean or standard deviation) 

 Date of Entry—YR (no mean or standard deviation) 

 Date of Entry—MO (no mean or standard deviation) 

 Date of Entry—DAY (no mean or standard deviation) 

 AFQT Test Group (no mean or standard deviation) 

 AFQT Percentile 

 Standardized Subtest Scores 

 GS AR 

 WK PC 

 NO CS 

 AS MK 

 MC EI 

 VE 

Army ASVAB Composites 

 GT GM 

 EL CL 

 MM SC 

 CO FA 

 OF ST 

1 Enclosed floppy contains data for three commissioned officers. It is understood that no AFQT and ASVAB 

Scaled Subtests or Composites will exist for those soldiers. 
111

3. For all Active duty soldiers (Army) holding MOS 96B, please provide N (population size), means and standard 

deviation of: 

 Age (use Date of Birth on file and 1 Mar 99 as current date in computation) 

 Time in Service (use Date of Entry on file and 1 Mar 99 as current date in computation) 

 AFQT Percentile 

 All Standardized Subtest Scores (referenced in para 2 above) 

 All Army ASVAB Composites (referenced in para 2 above) 

4. For all Active duty soldiers (Army) holding MOS 96B, please provide frequency distributions for: 

 Paygrade 

 Sex (Gender) 

 Race Ethnic 

 AFQT Test Group 

 Education with categories' definitions 

 Highest Year of Education with categories' definitions 

5. For all Active duty commissioned officers holding a 35D MOS, please provide, by rank (paygrade), N 

(population size) and frequency distributions for: 

 Sex (Gender) 

 Race Ethnic 

 Education (variable in column 11 on the Active Duty...file) 

 Highest Year of Education (variable in column 24 of the Active Duty...file) 

 Paygrade 

6. For all Active duty commissioned officers holding a 35D MOS, please provide, by rank (paygrade), N 

(population size) and mean and standard deviations of 

 Age (use Date of Birth and 1 Mar 99 as current date in computation) 

 Time in Service (use Date of Entry on file and 1 Mar 99 as current date in computation) 

7. File on floppy has been created with MS Word and saved in two forms: a) Text and b) Rich Text format. 

Please direct any questions concerning this request to Dr. Otto Heuckeroth, DSN 738-9377/9572, commercial (254) 

288-9377/9572. 

8. Please forward all output to: 

 Army Research Laboratory 

 ATTN: AMSRL-HR-MV (Dr. Heuckeroth) 

 HQ TEXCOM 

 91012 Station Ave 

 Fort Hood, TX 76544-5073 

Fax, if convenient, to DSN 738-1691, commercial (254) 288-1691. 

 OTTO H. HEUCKEROTH 

 Research Psychologist 

Encl 

Floppy disk (flat file) 

95-OT-1140A
112

INTENTIONALLY LEFT BLANK.
113

Appendix B. Sample Representativeness of Soldiers Participating in the 

March 1999 DT/OT RWS LUTE

 

This appendix appears in its original form, without editorial change. 
114

Sample Representativeness of Soldiers Participating in the March 1999 DT/OT RWS LUTE 

INTRODUCTION 

 Between 16 and 19 March 1999, enlisted soldiers holding an MOS 96B from the 3rd BDE of the 1st CD 

and B Co. of the 104th MI BN participated in a Tester controlled DT/OT event. In addition to these 10 enlisted 

soldiers, these three commissioned officers--a MAJ and a 1LT from the 3rd BDE of the 1st Co. can one 2LT from B 

Co. of the 104th MI BN. 

METHOD 

 In addition to the 14 instruments used by ARL to facilitate the MANPRINT assessment--from individual 

RWS task classroom training assessment through the MANPRINT assessment during the Record test--data on 

selected demographic and Armed Services Vocational Aptitude Battery (ASVAB) data were obtained for enlisted 

personnel; only selected demographic data were obtained for the participating commissioned officers.3 Variables 

used to make sample representativeness comparisons for the enlisted soldiers include: 1) Each of the 11 ASVAB 

Standardized Subtest Scores; 2) Armed Force Qualification Test (AFQT) percentile; 3) Each of the 10 ASVAB 

Composites; 4 Age; 5) Time in Service; 6) Gender; 7) Race/Ethnicity and 8) Education. Variables used to make 

sample representativeness judgments about participating commissioned officers include: 1) Gender; 2) 

Race/Ethnicity; 3) Education; 4) Time in Service; and 5) Age. Coordination with DMDC indicated that Composite 

scores in the Active Duty File were suspect in part, due to file formatting problems. In some cases, some Subtest 

Scores, from which Composites were computed, were missing; these records were deleted in the descriptive 

statistics presented for comparisons involving Composites. Finally, Composite data for Active Duty soldiers who 

entered the service prior to 1981 were not normalized such that they are comparable to ASVAB Composites data 

computed after 1980. For every Composite of about 500 Active Duty soldiers in the pre '81 aggregate population 

(MOS 96B, 96D, 98C), the mean normalized Composites are smaller and variances larger than those for comparable 

Active Duty soldiers taking the ASVAB after 1980. To reduce ambiguity in interpretation, only the Composites for 

that portion of the aggregate Active Duty population who took the ASVAB after 1980 are reported in Table 1. 

RESULTS AND DISCUSSION 

Sample Representativeness Comparisons for Enlisted MOS 96B Soldiers Participating in the RWS Enclave During 

the March 1999 DT/OT LUTE. 

Table 1 provides the means and standard deviations of ASVAB Scaled Subtests, 

Composites and Armed Forces Qualification Test (AFQT) percentile for active duty soldiers holding MOS 96B; 

comparable data are provided for the 10 MOS 96B who participated as RWS Analysts during the DT/OT test in 

March 1999. Also included in this table are statistics and parameters for soldier time in service and age. 

 

3

The source of both ASVAB and the selected demographic characteristics was from the Defense Manpower Data Center 

(DMDC) MEPCOM and Active Duty files. 
115

Table 1 

Comparison of Mean and Variance of Armed Services Vocational Aptitude Battery (ASVAB) Scaled Subtests and 

Composites, and Age and Time In Service of the Population of Active Duty Enlisted Soldiers Holding MOS 96B 

and Analysts Operating Remote Work Stations (RWS) During the March 1999 DT/OT LUTE 

POPULATION OF ACTIVE DUTY 

SOLDIERS HOLDING 

MOS 96B, 96D OR 98C 

PARTICIPANTS IN THE 

DECEMBER 1998 WFX 

SERVING AS RWS ANALYSTS 

MEANS 

SIGNIFICANCE 

TESTING 

VARIANC

E SIGNIFICANCE 

TESTING 

Variable N Mean Std Dev N Mean Std Dev t Test F

AFQTP 2406 72.22 15.78 10 80.50 13.99 1.66 1.27 

GS 2991 56.43 6.55 10 58.60 8.33 0.82 1.62 

AR 2991 56.08 6.77 10 59.40 5.93 1.77 1.30 

WK 2991 55.69 5.34 10 56.30 4.92 0.39 1.18 

PC 2991 55.56 5.10 10 57.10 6.23 0.78 1.49 

NO 2991 55.35 6.47 10 54.30 5.72 0.58 1.28 

CS 2991 54.52 7.56 10 57.30 6.11 1.44 1.53 

AS 2991 51.80 7.95 10 49.50 9.58 0.76 1.45 

MK 2991 57.31 7.15 10 62.90 3.21 5.46*** 4.96*** 

MC 2991 55.87 7.46 10 58.70 3.68 2.41* 4.11** 

EI 2991 53.38 7.42 10 50.50 10.01 0.91 1.82 

VE 2991 55.38 5.64 10 57.00 5.21 0.98 1.17 

GT 2400 113.55 9.54 10 117.90 10.09 1.44 1.12 

GM 2400 112.04 11.18 10 112.50 15.25 0.13 1.86 

EL 2406 114.10 10.54 10 117.80 12.02 1.11 1.30 

CL 2400 114.93 9.78 10 121.30 7.86 2.06* 1.55 

MM 2400 110.70 11.60 10 108.00 10.82 0.73 1.15 

SC 2400 112.06 10.48 10 114.40 10.72 0.70 1.05 

CO 2400 112.26 11.69 10 115.80 9.15 0.96 1.63 

FA 2400 115.46 11.11 10 123.30 7.21 3.42*** 2.37 

OF 2400 112.55 9.69 10 112.10 8.08 0.15 1.44 

ST 2400 116.01 9.52 10 121.40 8.26 1.79 1.33 

TIME IN 

SERVICE 

2406 77.40 58.44 10 45.00 50.40 1.75 1.34 

AGE (yrs) 2406 27.13 5.99 10 25.57 6.58 0.82 1.21 

*** P < .01 

 ** P < .025 

 * P < .05 

Tests for equality of variances were compared with F tests; comparisons of means were assessed by use of t

tests. Comparison of soldier participants in the WFX LUTE with the population distributions of gender, race and 

education were assessed by comparison of observed and expected frequencies (see Tables 2 - 5). As the expected 

frequencies in at least one category were less than 5, 2 goodness-of-fit tests would be of questionable validity. 

“Inter-ocular” comparisons of observed and expected frequencies were performed. 
116

Table 2 

Observed and Expected Frequency of Enlisted Soldiers Participating as RWS Analysts During the March 1999 

DT/OT LUTE by Categories of Gender 

Number of 

Active Duty Soldiers 

(MOS 96B) Expected 

Proportions 

Observed and Expected Frequency of RWS 

Test Participants 

Category of Gender 

 

Observed Expected 

1932 .805 7 8.05 MALE 

468 

. 

195 3 1.95 FEMALE 

2400 

 

10 

 

TOTAL 

Table 3 

Observed and Expected Frequency of Soldiers Participating as RWS Analysts During the March 1999 DT/OIT 

LUTE by Categories of Race 

Number of Active 

Duty Soldiers 

(MOS 96B) Expected 

Proportions 

Observed and Expected Frequency of RWS 

Test Participants 

Category of Race 

 

Observed Expected 

1776 .740 8 7.4 WHITE 

379 .158 1 1.58 BLACK 

112 .047 1 .47 HISPANIC 

133 .055 0 .55 OTHER 

2400 

 

10 

 

TOTAL 
117

Table 4 

Observed and Expected Frequency of Enlisted Soldiers participating as RWS Analysts During the March 1999 

DT/OT LUTE by Categories of Education 

Number of Active 

Duty Soldiers 

(MOS 96B) Expected 

Proportions 

Observed and Expected Frequency of RWS 

Test Participants 

Category of 

Education 

 

Observed Expected 

2047 .853 6 8.53 

HS Diploma or GED 

79 .033 1 .33 2 Yrs College 

59 .025 0 .25 

3-4 Yrs College, No 

Diploma 

183 .076 3 .76 

College Degree (4 

Yrs) 

14 .006 0 .06 Masters Degree 

1 .000 0 .00 Doctorate 

17 .007 0 .07 Unknown 

2400 

 

10 

 

Total 

Sample Representativeness of MOS 35D Commissioned Officers Participating in the RWS Enclave During the 

March 1999 DT/OT LUTE. 

 As the RWS enclave contained only three 35D MOS officers--a male MAJ as S2, a female LT1 as AS2 and 

a female LT2 as ACT chief--statistical comparisons were not feasible. To make an assessment of sample 

representativeness for officers with such small sample sizes, comparisons on gender, race and education variables 

are made by comparing the model frequency of active duty commissioned officers--for the ranks participating in the 

test--with the category value of the test participant (see Tables 5-8). It is assumed that commissioned officers of 

each rank used in the test were assigned to enclave functionality positions consistent with established MI doctrine. 

For time in service and age variables, means and standard deviations for each participating Active Duty Rank is 

shown together with the relevant information for each rank participating in the March 1999 RWS DT/OT LUTE (see 

Table 8). 
118

Table 5 

Frequency of Active Duty Commissioned Officers Holding MOS 35D in March 1999 by Gender1

Number of Active Duty Officers by 

Category 

Rank of Commissioned Officers Participating in the TEST 

 2LT 1LT MAJ 

Males 193 533 634* 

Females 105* 180* 86 1

Asterisks indicate the Gender of each MOS 35D test participant. 

Table 6 

Frequency of Active Duty Commissioned Officers Holding MOS 35D in March 1999 by Race/Ethnicity1

Number of Active Duty Officers by 

Category 

Rank of Commissioned Officers Participating in the TEST 

 2LT 1LT MAJ 

White 219* 548* 580* 

Black 40 81 67 

Hispanic 19 30 44 

Other 20 47 23 1

Asterisks indicated the Race of each MOS 35D test participant. 
119

Table 7 

Frequency of Active Duty Commissioned Officers Holding MOS 35D in March 1999 by Education1

Number of Active Duty Officers by 

Category 

Rank of Commissioned Officers Participating in the TEST 

 2LT 1LT MAJ 

HS/GED 3 3 0 

2 Yrs College 3 9 1 

BS/BA 255* 605* 372* 

MS/MA/1st Profn 6 25 337 

Doctorate 0 0 1 1

Asterisks indicated the level of education attained by the Commissioned Officer at the indicated rank who 

participated in the March 1999 DT/OT LUTE. 

Table 8 

Summary Descriptive Statistics for Age and Time in Service for 2LTs, 1LTs and MAJs Holding MOS 35D1

 2LT 1LT MAJ 

AGE 

N 298 710 719 

 Mean 25.98 (22.75) 27.11 (23.83) 38.08 (39.17) 

 Std Dev. 2.62 2.84 3.18 

 

Time in Service 

N 298 713 720 

 Mean 2.46 (0.75) 4.31 (1.83) 14.80 (13.42) 

Std Dev. 2.58 2.67 2.56 1

Number in parentheses is the Age and Time in Service of Commissioned Officers who participated in the march 

1999 DT/OT LUTE. 
120

CONCLUSIONS 

Enlisted Soldier Participants 

 Compared to the population of Active Duty MOS 96B soldiers, test participants were 

significantly more homogeneous and had significantly higher aptitude scores for Mathematics 

Knowledge (MK) and Mechanical Comprehension (MC) Subtests compared to the population of 

Active Duty MOS 96B (see Table 1). Both of these Subtest raw scores are used to compute 

ASVAB Composites FA and ST. While the enlisted test participants also show the same 

significance pattern for the FA Composite, differences for the ST Composite--a Composite for 

which entry into MOS 96B is set at 105 as a minimum (AR 611-201)--while in the same 

direction, are not significant. In addition, mean CL aptitude is significantly greater for MOS 96B 

for test participants than for the population from which they were selected. These differences 

might, in part, be attributed to the fact that enlisted MOS 96B test participants were slightly more 

educated (see Table 4) than the population from which they were selected. No marked 

differences in gender (Table 2) or Race (Table 3) distributions were noted. These findings 

together with the observation that the differences for many of the other Subtests that the enlisted 

MOS 96B test participants are slightly superior in aptitude and education compared to target 

audience population from which they were selected. Had these differences not existed, it is this 

analyst's judgment that the training (need to integrate conceptual MOS TTPs training with 

training on use of automated software) and software problems exhibited would have been even 

more severe. With modification in the training (as recommended) and correction of software 

problems, it is this analyst's judgment that with a more representative set of enlisted participants 

the automated RWS function would be demonstrated as effective and suitable, i.e., the slight lack 

of sample representativeness is much less important to the overall system evaluation than the 

training deficiencies and HFE (software) problems evidenced.

Commissioned Officers Participants 

 Apart from the fact that the Assistant S2 (AS2) and ACT Chief were younger and with 

less experience than the population of MOS 35D commissioned officers from which they were 

selected, no concern about representativeness is evident. Both AS2 and the ACT Chief appeared 

quite competent and seemed to work well with the S2 and enlisted analysts. As with the enlisted 

analysts test participants, any slight unrepresentativeness noted is much less important to the 

evaluation assessment than addressing training deficiencies and correcting software problems 

evidenced.
121

Appendix C. Human Factors Engineering Questionnaire for an Electronic 

Equipment Test Facility (EETF)

 

This appendix appears in its original form, without editorial change. 
122

HUMAN FACTORS QUESTIONNAIRE 

ELECTRONIC TEST FACILITY OPERATOR (35C20) 

 NAME_______________________ DATE______________ 

The purpose of the following questionnaire is to obtain your opinions 

 and observations about the adequacy of the AN/MSM 105 Electronic Test 

 Facility from an operator's point of view. Take as much time as you feel 

 is necessary to accurately complete the questionnaire. The administrator 

 will answer any questions you have. 
123

I. ENVIRONMENT 

Completely Adequate Mostly Adequate Borderline Mostly Inadequate Completely Inadequate 

A. Using the scale to the right 

 indicate with a check mark (√) 

 to indicate the adequacy of the 

 following environmental 

 conditions in the ETF: 

 

1. Temperature ___ ___ ___ ___ ___ 

2. Ventilation ___ ___ ___ ___ ___ 

3. Noise ___ ___ ___ ___ ___ 

4. Vibration ___ ___ ___ ___ ___ 

5. Illumination ___ ___ ___ ___ ___ 

B. Explanation of Borderline, Mostly Inadequate and Completely Inadequate responses 

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________ 
124

II. EQUIPMENT CHARACTERISTICS 

Completely Adequate Mostly Adequate Borderline Mostly Inadequate Completely Inadequate 

 A. VIDEO DISPLAY UNIT 

 Using the scale to the right 

 indicate with a check mark (√) 

 to indicate the adequacy of the 

 following environmental 

 conditions in the ETF: 

 

1. DISPLAY 

 a. Display brightness ___ ___ ___ ___ ___ 

 b. Absence of glare ___ ___ ___ ___ ___ 

 c. Absence of flicker ___ ___ ___ ___ ___ 

 d. Letter discrimination ___ ___ ___ ___ ___ 

 e. Viewing distance ___ ___ ___ ___ ___ 

 f. Angle of view ___ ___ ___ ___ ___ 

 g. Location of display ___ ___ ___ ___ ___ 

 h. Other (specify)________ 

 ___________________ 

___ ___ ___ ___ ___ 

2. INDICATOR PANEL 

 

 

 a. Display brightness ___ ___ ___ ___ ___ 

 b. Absence of glare ___ ___ ___ ___ ___ 

 c. Absence of flicker ___ ___ ___ ___ ___ 

 d. Viewing distance ___ ___ ___ ___ ___ 

 e. Angle of view ___ ___ ___ ___ ___ 

 f. Correct labels ___ ___ ___ ___ ___ 

 g. Location of indicators ___ ___ ___ ___ ___ 

 h. Other (specify)_____ 

 _________________ 

___ ___ ___ ___ ___ 
125

3. KEYBOARD AND CONTROLS 

Completely Adequate Mostly Adequate Borderline Mostly Inadequate Completely Inadequate 

 a. Size ___ ___ ___ ___ ___ 

 

 b. Shape ___ ___ ___ ___ ___ 

 c. Spacing between controls ___ ___ ___ ___ ___ 

 d. Resistance (too easy to 

 turn or push, or too hard 

 to turn or push) 

___ ___ ___ ___ ___ 

 

 e. Correct/Complete labels ___ ___ ___ ___ ___ 

 f. Understandable labels ___ ___ ___ ___ ___ 

 g. Size of labels ___ ___ ___ ___ ___ 

 h. Location of labels ___ ___ ___ ___ ___ 

 i. Absence of unrelated or 

 confusing markings 

___ ___ ___ ___ ___ 

 

 j. Visibility of controls ___ ___ ___ ___ ___ 

 k. Angle of view ___ ___ ___ ___ ___ 

 l. Location of critical controls ___ ___ ___ ___ ___ 

 

 m. Reach distance of 

 critical controls ___ ___ ___ ___ ___ 

 

 n. Location of non-critical

 controls ___ ___ ___ ___ ___ 

 

 o. Reach distance of 

 non-critical controls ___ ___ ___ ___ ___ 

 

 p. Functional grouping (controls 

 with related functions are 

 grouped together) 

___ ___ ___ ___ ___ 
126

Completely Adequate Mostly Adequate Borderline Mostly Inadequate Completely Inadequate 

 q. Control type (type of 

 control is appropriate 

 for type of function) 

___ ___ ___ ___ ___ 

 r. Other (specify)_____ 

 _________________ 

___ ___ ___ ___ ___ 

B. Explanation of Borderline, Mostly Inadequate and Completely Inadequate responses 

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________
127

Completely Adequate Mostly Adequate Borderline Mostly Inadequate Completely Inadequate 

B. LINE PRINTER 

 Using the scale to the right 

 indicate with a check mark (√) 

 to indicate the adequacy of the 

 following environmental 

 conditions in the ETF: 

 

1. INDICATOR LIGHTS 

 a. Brightness ___ ___ ___ ___ ___ 

 

 b. Absence of glare ___ ___ ___ ___ ___ 

 

 c. Absence of flicker ___ ___ ___ ___ ___ 

 

 d. Viewing distance ___ ___ ___ ___ ___ 

 

 e. Angle of view ___ ___ ___ ___ ___ 

 

 f. Correct labels ___ ___ ___ ___ ___ 

 

 g. Location of indicators ___ ___ ___ ___ ___ 

 

 h. Indicators inform you of what 

 you need to know 

 ___________________ 

___ ___ ___ ___ ___ 

 (1) in a timely manner ___ `___ ___ ___ ___ 

 

 (2) with enough precision ___ ___ ___ ___ ___ 

 

 (3) with relevant information ___ ___ ___ ___ ___ 

 

 i. Other (specify)_______________ 

 __________________________ ___ ___ ___ ___ ___ 

2. CONTROLS 

 

 a. Size ___ ___ ___ ___ ___ 

 

 b. Shape ___ ___ ___ ___ ___ 

 

 c. Spacing between controls ___ ___ ___ ___ ___ 

 

 d. Resistance (too easy to 

 turn or push, or too hard 

 to turn or push) 

___ ___ ___ ___ ___ 
128

Completely Adequate Mostly Adequate Borderline Mostly Inadequate Completely Inadequate 

 e. Correct/Complete labels ___ ___ ___ ___ ___ 

 

 f. Understandable labels ___ ___ ___ ___ ___ 

 

 g. Size of labels ___ ___ ___ ___ ___ 

 

 h. Location of labels ___ ___ ___ ___ ___ 

 

 i. Absence of unrelated or 

 confusing markings 

___ ___ ___ ___ ___ 

 

 j. Visibility of controls ___ ___ ___ ___ ___ 

 

 k. Angle of view ___ ___ ___ ___ ___ 

 

 l. Location of critical controls ___ ___ ___ ___ ___ 

 

 m. Reach distance of 

 critical controls ___ ___ ___ ___ ___ 

 

 n. Location of non-critical

 controls ___ ___ ___ ___ ___ 

 

 o. Reach distance of 

 non-critical controls ___ ___ ___ ___ ___ 

 

 p. Functional grouping (controls 

 with related functions are 

 grouped together) 

___ ___ ___ ___ ___ 

 

 q. Control type (type of control 

 is appropriate for type of 

 function) 

___ ___ ___ ___ ___ 

 

 r. Other (Specify)______________ 

 __________________________ ___ ___ ___ ___ ___ 
129

B. Explanation of Borderline, Mostly Inadequate and Completely Inadequate responses 

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________ 

_____________________________________________________________________________________________ 

_____________________________________________________________________________________________

_____________________________________________________________________________________________ 
130Completely Adequate 

Mostly Adequate Borderline Mostly Inadequate Completely Inadequate 

C. CONTROL STATION 

 Using the scale to the right 

 indicate with a check mark (√) 

 to indicate the adequacy of the 

 following environmental 

 conditions in the ETF: 

 

1. COUNTERS AND INDICATORS 

 a. Brightness ___ ___ ___ ___ ___ 

 

 b. Absence of glare ___ ___ ___ ___ ___ 

 c. Absence of flicker ___ ___ ___ ___ ___ 

 d. Viewing distance ___ ___ ___ ___ ___ 

 e. Angle of view ___ ___ ___ ___ ___ 

 f. Correct labels ___ ___ ___ ___ ___ 

 g. Location of indicators ___ ___ ___ ___ ___ 

 h. Counters and Indicators inform 

 you of what you need to know ___ ___ ___ ___ ___ 

 (1) in a timely manner ___ `___ ___ ___ ___ 

 

 (2) with enough precision ___ ___ ___ ___ ___ 

 (3) with relevant information ___ ___ ___ ___ ___ 

 i. Other 

(specify)_______________ 

 __________________________ 

___ ___ ___ ___ ___ 

2. CONTROLS 

 

 a. Size ___ ___ ___ ___ ___ 

 b. Shape ___ ___ ___ ___ ___ 

 c. Spacing between controls ___ ___ ___ ___ ___ 

 d. Resistance (too easy to 

 turn or push, or too hard 

 to turn or push) 

___ ___ ___ ___ ___ 
131 Completely Adequate 

Mostly Adequate 

Borderline 

Mostly Inadequate 

Completely Inadequate 

 e. Correct/Complete labels ___ ___ ___ ___ ___ 

 

 f. Understandable labels ___ ___ ___ ___ ___ 

 g. Size of labels ___ ___ ___ ___ ___ 

 h. Location of labels ___ ___ ___ ___ ___ 

 i. Absence of unrelated or 

 confusing markings 

___ ___ ___ ___ ___ 

 

 j. Visibility of controls ___ ___ ___ ___ ___ 

 k. Angle of view ___ ___ ___ ___ ___ 

 l. Location of critical controls ___ ___ ___ ___ ___ 

 

 m. Reach distance of 

 critical controls ___ ___ ___ ___ ___ 

 

 n. Location of non-critical

 controls ___ ___ ___ ___ ___ 

 

 o. Reach distance of 

 non-critical controls ___ ___ ___ ___ ___ 

 

 p. Functional grouping (controls 

 with related functions are 

 grouped together) 

___ ___ ___ ___ ___ 

 

 q. Control type (type of control 

 is appropriate for type of 

 function) 

___ ___ ___ ___ ___ 

 

 r. Other (Specify)______________ 

 __________________________ ___ ___ ___ ___ ___ 
132

3. Explanation of Borderline, Mostly Inadequate and Completely Inadequate responses 

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________ 
133

Completely Adequate Mostly Adequate Borderline Mostly Inadequate Completely Inadequate 

D. DC POWER STATION 

 Using the scale to the right 

 indicate with a check mark (√) 

 to indicate the adequacy of the 

 following environmental 

 conditions in the ETF: 

 

1. METERS AND INDICATORS 

 a. Brightness ___ ___ ___ ___ ___ 

 

 b. Absence of glare ___ ___ ___ ___ ___ 

 c. Absence of flicker ___ ___ ___ ___ ___ 

 d. Viewing distance ___ ___ ___ ___ ___ 

 e. Angle of view ___ ___ ___ ___ ___ 

 f. Correct labels ___ ___ ___ ___ ___ 

 g. Location of meters and 

 indicators 

___ ___ ___ ___ ___ 

 

 h. Meters and Indicators inform 

 you of what you need to know 

 ___________________ 

___ ___ ___ ___ ___ 

 (1) in a timely manner ___ `___ ___ ___ ___ 

 

 (2) with enough precision ___ ___ ___ ___ ___ 

 (3) with relevant information ___ ___ ___ ___ ___ 

 i. Other 

 (specify)_______________ 

 __________________________ 

___ ___ ___ ___ ___ 

2. CONTROLS 

 

 a. Size ___ ___ ___ ___ ___ 

 b. Shape ___ ___ ___ ___ ___ 

 c. Spacing between controls ___ ___ ___ ___ ___ 

 d. Resistance (too easy to 

 turn or push, or too hard 

 to turn or push) 

___ ___ ___ ___ ___ 

 

 e. Correct/Complete labels ___ ___ ___ ___ ___ 
134

Completely Adequate 

Mostly Adequate Borderline Mostly Inadequate Completely Inadequate 

 f. Understandable labels ___ ___ ___ ___ ___ 

 

 g. Size of labels ___ ___ ___ ___ ___ 

 h. Location of labels ___ ___ ___ ___ ___ 

 i. Absence of unrelated or 

 confusing markings 

___ ___ ___ ___ ___ 

 

 j. Visibility of controls ___ ___ ___ ___ ___ 

 k. Angle of view ___ ___ ___ ___ ___ 

 l. Location of critical controls ___ ___ ___ ___ ___ 

 

 m. Reach distance of 

 critical controls ___ ___ ___ ___ ___ 

 

 n. Location of non-critical

 controls ___ ___ ___ ___ ___ 

 

 o. Reach distance of 

 non-critical controls ___ ___ ___ ___ ___ 

 

 p. Functional grouping (controls 

 with related functions are 

 grouped together) 

___ ___ ___ ___ ___ 

 

 q. Control type (type of control 

 is appropriate for type of 

 function) 

___ ___ ___ ___ ___ 

 

 r. Other (Specify)______________ 

 __________________________ ___ ___ ___ ___ ___ 
135

3. Explanation of Borderline, Mostly Inadequate and Completely Inadequate responses 

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________ 

_____________________________________________________________________________________________

_____________________________________________________________________________________________ 
136Completely Adequate 

Mostly Adequate Borderline Mostly Inadequate Completely Inadequate 

E. TAPE STATION 

 Using the scale to the right 

 indicate with a check mark (√) 

 to indicate the adequacy of the 

 following environmental 

 conditions in the ETF: 

 

1. INDICATOR LIGHTS 

 a. Brightness ___ ___ ___ ___ ___ 

 

 b. Absence of glare ___ ___ ___ ___ ___ 

 c. Absence of flicker ___ ___ ___ ___ ___ 

 d. Viewing distance ___ ___ ___ ___ ___ 

 e. Angle of view ___ ___ ___ ___ ___ 

 f. Correct labels ___ ___ ___ ___ ___ 

 g. Location of indicators ___ ___ ___ ___ ___ 

 h. Indicators inform you of what 

 you need to know 

 ___________________ 

___ ___ ___ ___ ___ 

 (1) in a timely manner ___ `___ ___ ___ ___ 

 

 (2) with enough precision ___ ___ ___ ___ ___ 

 (3) with relevant information ___ ___ ___ ___ ___ 

 i. Other 

(specify)_______________ 

 __________________________ 

___ ___ ___ ___ ___ 

2. CONTROLS 

 a. Size ___ ___ ___ ___ ___ 

 b. Shape ___ ___ ___ ___ ___ 

 c. Spacing between controls ___ ___ ___ ___ ___ 

 d. Resistance (too easy to 

 turn or push, or too hard 

 to turn or push) 

___ ___ ___ ___ ___ 
137Completely Adequate 

Mostly Adequate Borderline Mostly Inadequate Completely Inadequate 

 e. Correct/Complete labels ___ ___ ___ ___ ___ 

 

 f. Understandable labels ___ ___ ___ ___ ___ 

 g. Size of labels ___ ___ ___ ___ ___ 

 h. Location of labels ___ ___ ___ ___ ___ 

 i. Absence of unrelated or 

 confusing markings 

___ ___ ___ ___ ___ 

 

 j. Visibility of controls ___ ___ ___ ___ ___ 

 k. Angle of view ___ ___ ___ ___ ___ 

 l. Location of critical controls ___ ___ ___ ___ ___ 

 

 m. Reach distance of 

 critical controls ___ ___ ___ ___ ___ 

 

 n. Location of non-critical

 controls ___ ___ ___ ___ ___ 

 

 o. Reach distance of 

 non-critical controls ___ ___ ___ ___ ___ 

 

 p. Functional grouping (controls 

 with related functions are 

 grouped together) 

___ ___ ___ ___ ___ 

 

 q. Control type (type of control 

 is appropriate for type of 

 function) 

___ ___ ___ ___ ___ 

 

 r. Other (Specify)______________ 

 __________________________ ___ ___ ___ ___ ___ 
138

3. Explanation of Borderline, Mostly Inadequate and Completely Inadequate responses 

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________ 
139

F. UNIT UNDER TEST (UUT) STATION 

Using the scale to the right indicate with 

a check mark (√) how adequate the UUT 

Station is in each of the following 

areas: 

1. CONTROLS 

Completely Adequate Mostly Adequate Borderline Mostly Inadequate Completely Inadequate 

 a. Size ___ ___ ___ ___ ___ 

 

 b. Shape ___ ___ ___ ___ ___ 

 c. Spacing between controls ___ ___ ___ ___ ___ 

 d. Resistance (too easy to 

 turn or push, or too hard 

 to turn or push) 

___ ___ ___ ___ ___ 

 

 e. Correct/Complete labels ___ ___ ___ ___ ___ 

 f. Understandable labels ___ ___ ___ ___ ___ 

 g. Size of labels ___ ___ ___ ___ ___ 

 h. Location of labels ___ ___ ___ ___ ___ 

 i. Absence of unrelated or 

 confusing markings 

___ ___ ___ ___ ___ 

 

 j. Visibility of controls ___ ___ ___ ___ ___ 

 k. Angle of view ___ ___ ___ ___ ___ 

 l. Location of critical controls ___ ___ ___ ___ ___ 

 

 m. Reach distance of 

 critical controls ___ ___ ___ ___ ___ 

 

 n. Location of non-critical

 controls ___ ___ ___ ___ ___ 

 

 o. Reach distance of 

 non-critical controls ___ ___ ___ ___ ___ 

 

 p. Functional grouping (controls 

 with related functions are 

 grouped together) 

___ ___ ___ ___ ___ 
140

Completely Adequate Mostly Adequate Borderline Mostly Inadequate Completely Inadequate 

 q. Control type (type of 

 control is appropriate 

 for type of function) 

___ ___ ___ ___ ___ 

 

 r. Other (specify)_____ 

 _________________ 

___ ___ ___ ___ ___ 

B. Explanation of Borderline, Mostly Inadequate and Completely Inadequate responses 

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________ 
141

G. PROGRAMMABLE INTERFACE UNIT 

(PIU) STATION 

Using the scale to the right indicate 

with a check mark (√) how adequate 

the PIU) Station is in each of the 

following areas: 

1. CONTROLS 

Completely Adequate Mostly Adequate Borderline Mostly Inadequate Completely Inadequate 

 a. Size ___ ___ ___ ___ ___ 

 

 b. Shape ___ ___ ___ ___ ___ 

 c. Spacing between controls ___ ___ ___ ___ ___ 

 d. Resistance (too easy to 

 turn or push, or too hard 

 to turn or push) 

___ ___ ___ ___ ___ 

 

 e. Correct/Complete labels ___ ___ ___ ___ ___ 

 f. Understandable labels ___ ___ ___ ___ ___ 

 g. Size of labels ___ ___ ___ ___ ___ 

 h. Location of labels ___ ___ ___ ___ ___ 

 i. Absence of unrelated or 

 confusing markings 

___ ___ ___ ___ ___ 

 

 j. Visibility of controls ___ ___ ___ ___ ___ 

 k. Angle of view ___ ___ ___ ___ ___ 

 l. Location of critical controls ___ ___ ___ ___ ___ 

 

 m. Reach distance of 

 critical controls ___ ___ ___ ___ ___ 

 

 n. Location of non-critical

 controls ___ ___ ___ ___ ___ 

 

 o. Reach distance of 

 non-critical controls ___ ___ ___ ___ ___ 

 

 p. Functional grouping (controls 

 with related functions are 

 grouped together) 

___ ___ ___ ___ ___ 
142 Completely Adequate 

Mostly Adequate Borderline Mostly Inadequate Completely Inadequate 

 q. Control type (type of 

 control is appropriate 

 for type of function) 

___ ___ ___ ___ ___ 

 

 r. Other 

(specify)_____ 

 _________________ 

___ ___ ___ ___ ___ 

2. Explanation of Borderline, Mostly Inadequate and Completely Inadequate responses 

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________ 
143

III. OVERALL CONFIGURATION OF ETF 

 A. EQUIPMENT LOCATION 

 Using the scale to the right 

 indicate with a check mark (√) 

 the adequacy of the location

 within the ETF of each of the 

 following pieces of equipment: 

Completely Adequate Mostly Adequate Borderline Mostly Inadequate Completely Inadequate 

1. Video Display Terminal ___ ___ ___ ___ ___ 

2. Printer ___ ___ ___ ___ ___ 

3. UUT Station ___ ___ ___ ___ ___ 

4. PIU Station ___ ___ ___ ___ ___ 

5. DC Power Station ___ ___ ___ ___ ___ 

6. Control Station ___ ___ ___ ___ ___ 

7. Tape Station ___ ___ ___ ___ ___ 

8. Storage Racks ___ ___ ___ ___ ___ 

9. Storage Cabinets ___ ___ ___ ___ ___ 

10. Work Bench ___ ___ ___ ___ ___ 

11. Work Desk ___ ___ ___ ___ ___ 

12. Cables ___ ___ ___ ___ ___ 

13. Other (specify)_________________ 

 _____________________________ 

___ ___ ___ ___ ___ 

14. Explanation of Borderline, Mostly Inadequate, and Completely Inadequate responses. 

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________ 
144

B. WORKSPACE 

 Using the scale to the right 

 indicate with a check mark √) 

 the adequacy of the Workspace 

 and Storage Space within the ETF 

Completely Adequate Mostly Adequate Borderline Mostly Inadequate Completely Inadequate 

1. Amount of workspace ___ ___ ___ ___ ___ 

2. Amount of storage space 

 for tools 

___ ___ ___ ___ ___ 

3. Amount of storage space 

 for manuals 

___ ___ ___ ___ ___ 

4. Amount of storage space 

 for personal gear 

___ ___ ___ ___ ___ 

5. Amount of storage space 

 for other items _______________ 

___ ___ ___ ___ ___ 

6. Explanation of Borderline, Mostly Inadequate, and Completely Inadequate responses. 

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________ 
145

IV. JOB PROCEDURES 

 Using the scale to the right 

 indicate with a check mark (√) 

 how easy of difficult it is to 

 perform each of the following 

 procedures: 

Very Easy Easy Borderline Difficult Very Difficult Did Not Perform 

 A. POWER-UP THE EQUIPMENT 

 1. Prepare the ETF for operation ___ ___ ___ ___ ___ ___ 

 2. Cable the system for operation ___ ___ ___ ___ ___ ___ 

 3. Power up to standby power-on ___ ___ ___ ___ ___ ___ 

 4. Power up to control and 

 display subsystem power-on 

___ ___ ___ ___ ___ ___ 

 5. Power up to full power-on ___ ___ ___ ___ ___ ___ 

 6. Recover from loss of power ___ ___ ___ ___ ___ ___ 

 7. Other procedures (specify) 

 ________________________ 

___ ___ ___ ___ ___ ___ 

8. Explanation of Borderline, Difficult, and Very Difficult 

____________________________________________________________________________________________

____________________________________________________________________________________________

____________________________________________________________________________________________

____________________________________________________________________________________________

____________________________________________________________________________________________

____________________________________________________________________________________________

____________________________________________________________________________________________

____________________________________________________________________________________________

____________________________________________________________________________________________

____________________________________________________________________________________________

____________________________________________________________________________________________ 
146

Ver

y Eas

y

Eas

y

BorderlineDifficult

Ver

y Difficult

Did Not Perform 

 B. MAGNETIC TAPE AND DISK 

 CARTRIDGE HANDLING 

 

1. Load the magnetic tape ___ ___ ___ ___ ___ ___ 

2. Unload the magnetic tape ___ ___ ___ ___ ___ ___ 

3. Install the disk cartridge ___ ___ ___ ___ ___ ___ 

4. Remove the disk cartridge 

│

___ ___ ___ ___ ___ ___ 

5. PREPARE A NEW DISK FOR USE ___ ___ ___ ___ ___ ___ 

 a. Format the disk ___ ___ ___ ___ ___ ___ 

 b. Initialize the disk ___ ___ ___ ___ ___ ___ 

 c. Install a system bootstrap 

 loader 

___ ___ ___ ___ ___ ___ 

 d. Transfer OS software 

 from tape to disk 

___ ___ ___ ___ ___ ___ 

 e. Build a duplicate disk ___ ___ ___ ___ ___ ___ 

6. Prepare an OS Diagnostic 

 Disk (DDOS) 

___ ___ ___ ___ ___ ___ 

7. Verify a disk ___ ___ ___ ___ ___ ___ 

8. Generate a system tape using 

 The OS 

___ ___ ___ ___ ___ ___ 

9. Other procedures (specify) 

 ____________________________ 

___ ___ ___ ___ ___ ___ 

8. Explanation of Borderline, Difficult, and Very Difficult 

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________ 

_____________________________________________________________________________________________
147

Very Easy Easy Borderline Difficult Very Difficult Did Not Perform 

 C. COMPUTER START-UP 

1. Normal start up from disk ___ ___ ___ ___ ___ ___ 

2. Start up from duplicate disk ___ ___ ___ ___ ___ ___ 

3. Start up from magnetic tape ___ ___ ___ ___ ___ ___ 

4. Restart system 

│

___ ___ ___ ___ ___ ___ 

5. Other procedures (specify) ___ ___ ___ ___ ___ ___ 

 6. Explanation of Borderline, Difficult, and Very Difficult 

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________ 
148Very Easy 

Easy Borderline Difficult Very Difficult Did Not Perform 

 D. DATA HANDLING 

1. System release ___ ___ ___ ___ ___ ___ 

2. Tape release ___ ___ ___ ___ ___ ___ 

3. Load UUT program from 

 tape to disk 

___ ___ ___ ___ ___ ___ 

4. Duplicate disk file 

│

___ ___ ___ ___ ___ ___ 

5. Delete disk file ___ ___ ___ ___ ___ ___ 

6. Search file name ___ ___ ___ ___ ___ ___ 

7. Display file content on VDT ___ ___ ___ ___ ___ ___ 

8. Print file content on printer ___ ___ ___ ___ ___ ___ 

9. Other procedures (specify) ___ ___ ___ ___ ___ ___ 

10. Explanation of Borderline, Difficult, and Very Difficult 

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________ 
149

Very Easy Easy Borderline Difficult Very Difficult Did Not Perform 

 E. TESTING THE UUT 

1. Test line replaceable units 

 (LRUs) using the 410 

___ ___ ___ ___ ___ ___ 

2. Identify the UUT to determine 

 the hardware and software 

 necessary to test it 

___ ___ ___ ___ ___ ___ 

3. Select and install the 

 correct memory pack 

___ ___ ___ ___ ___ ___ 

4. Determine the required mode 

 of equipment operation ___ ___ ___ ___ ___ ___ 

5. Connect the UUT to the 410 

 while following the display 

 diagrams on the VDT 

___ ___ ___ ___ ___ ___ 

6. Follow the TPS program 

 instructions ___ ___ ___ ___ ___ ___ 

7. Probe and test the UUT 

 while following the 

 computer instructions 

___ ___ ___ ___ ___ ___ 

8. Test UUTs with 465 DCT ___ ___ ___ ___ ___ ___ 

9. Other procedures (specify) ___ ___ ___ ___ ___ ___ 

10. Explanation of Borderline, Difficult, and Very Difficult 

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________ 
150 Ver

y Eas

y

Easy Borderline Difficult Very Difficult Did Not Perform 

 F. SELF TESTING AND ALIGNING 

1. Control and display subsystem 

 Testing 

 a. Perform the DDCS self test ___ ___ ___ ___ ___ ___ 

 b. Perform the DTOS self test ___ ___ ___ ___ ___ ___ 

 c. Perform the video terminal 

 self test ___ ___ ___ ___ ___ ___ 

 d. Perform the line printer self 

 test ___ ___ ___ ___ ___ ___ 

2. Self Testing 

 a. Perform the full ILSST.IC self 

 test ___ ___ ___ ___ ___ ___ 

 b. Perform the UUT Test Survey 

 Leader self test ___ ___ ___ ___ ___ ___ 

3. Systems Alignment Run the 

 SYSCAL Program ___ ___ ___ ___ ___ ___ 

 a. Manual Mode ___ ___ ___ ___ ___ ___ 

 b. Automatic Mode ___ ___ ___ ___ ___ ___ 

 c. Run all mode ___ ___ ___ ___ ___ ___ 

 d. Information Mode ___ ___ ___ ___ ___ ___ 

4. Perform the 465 DCT self test ___ ___ ___ ___ ___ ___ 

5. Other procedures (specify) ___ ___ ___ ___ ___ ___ 
151

6. Explanation of Borderline, Difficult, and Very Difficult 

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________ 
152

Very Easy Easy Borderline Difficult Very Difficult Did Not Perform 

 G. OPERATING THE TEST 

 EQUIPMENT 

 

1. Operate the computer using the 

 Command line printer (CLI) 

 

2. Store and maintain Test Program 

 Sets (TPSs) 

___ ___ ___ ___ ___ ___ 

3. Load paper in the line printer ___ ___ ___ ___ ___ ___ 

4. Use the oscilloscope 

___ ___ ___ ___ ___ ___ 

5. Other procedures (specify) 

___ ___ ___ ___ ___ ___ 

6. Explanation of Borderline, Difficult, and Very Difficult 

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________
153 Very Easy 

Easy Borderline Difficult Very Difficult Did Not Perform 

 H. POWER DOWN 

1. From full power-on to control and 

 display subsystem power-on 

2. From control and display sub- 

 system power-on to standby 

 power-on 

___ ___ ___ ___ ___ ___ 

3. From standby power-on to full 

 power-off 

___ ___ ___ ___ ___ ___ 

4. From any power-on state to 

 emergency power-off ___ ___ ___ ___ ___ ___ 

5. Other procedures (specify) 

___ ___ ___ ___ ___ ___ 

6. Explanation of Borderline, Difficult, and Very Difficult 

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________ 
154

Ver

y Eas

y

Eas

y

Borderline Difficult

Ver

y Difficult

Did Not Perform 

 I. ADMINISTRATIVE AND OTHER 

 PROCEDURES 

 

1. Assist the team chief in organizing 

 the workload for the 105 facility 

___ ___ ___ ___ ___ ___ 

2. Perform duties of team chief when 

 the team chief is absent 

___ ___ ___ ___ ___ ___ 

3. Keep up-to-date log on the 410 and 

 465 run times 

___ ___ ___ ___ ___ ___ 

4. Advise team chief of improperly 

 handled or abused items ___ ___ ___ ___ ___ ___ 

5. Route faulty modules through the 

 fault isolation and repair process ___ ___ ___ ___ ___ ___ 

6. Prepare and maintain the required 

 forms, records and reports ___ ___ ___ ___ ___ ___ 

7. Assist maintenance technicians in 

 in preventative maintenance and 

 and repair of the 410 

___ ___ ___ ___ ___ ___ 

8. Use the technical manuals to 

 operate and maintain the test 

 equipment 

___ ___ ___ ___ ___ ___ 

9. Other procedures (specify) ___ ___ ___ ___ ___ ___ 

6. Explanation of Borderline, Difficult, and Very Difficult 

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________ 
155

Very Easy Easy Borderline Difficult Very Difficult Did Not Perform 

 J. MAINTENANCE 

1. Perform preventative maintenance 

 checks and services on the ETF 

___ ___ ___ ___ ___ ___ 

2. Repair the 410 by replacing 

 assemblies and PCBs 

___ ___ ___ ___ ___ ___ 

3. Perform preventative maintenance 

 checks and services on the 

 oscilloscope 

___ ___ ___ ___ ___ ___ 

4. Assist the maintenance technician 

 in performing preventive 

 maintenance and repairs on the 

 410 

___ ___ ___ ___ ___ ___ 

5. Remove and replace a power 

 supply 

___ ___ ___ ___ ___ ___ 

6. Replace test operators panel 

 lamp 

___ ___ ___ ___ ___ ___ 

7. Perform maintenance on the 

 Individual stations 

___ ___ ___ ___ ___ ___ 

 a. DC Station ___ ___ ___ ___ ___ ___ 

 b. Control Station ___ ___ ___ ___ ___ ___ 

 c. UUT Station ___ ___ ___ ___ ___ ___ 

 d. PIU Station ___ ___ ___ ___ ___ ___ 

 e. Tape Station ___ ___ ___ ___ ___ ___ 

 f. 465 DCT ___ ___ ___ ___ ___ ___ 

8. Other procedures (specify) ___ ___ ___ ___ ___ ___ 
156

9. Explanation of Borderline, Difficult, and Very Difficult 

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________ 
157

Completely Adequate Mostly Adequate Borderline Mostly Inadequate Completely Inadequate 

V. COMPUTER SOFTWARE 

 Using the scale to the right, 

 indicate with a check mark (√) 

 the adequacy of the computer 

 software in each of the following 

 areas: 

 

 1. Type of user-software 

 dialogue 

___ ___ ___ ___ ___ 

 2. Consistency of display 

 formats 

___ ___ ___ ___ ___ 

 3. Clarity of prompts ___ ___ ___ ___ ___ 

 4. Usefulness of prompts ___ ___ ___ ___ ___ 

 5. Timeliness of prompts ___ ___ ___ ___ ___ 

 6. Clarity of error 

 messages 

___ ___ ___ ___ ___ 

 7. Usefulness of error 

 messages 

___ ___ ___ ___ ___ 

 8. Timeliness of error 

 messages 

___ ___ ___ ___ ___ 

 9. Clarity of test procedures 

 listed in the software 

___ ___ ___ ___ ___ 

10. Requirements for remembering 

 related information while 

 executing a program 

___ ___ ___ ___ ___ 

11. Other (specify) ___ ___ ___ ___ ___ 
158

12. Explanation of Borderline, Mostly Inadequate and Completely Inadequate 

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________ 
159

VI. SAFETY 

 Indicate with a check mark (√) whether you experienced or nearly 

 experienced any of the following safety hazards operating the ETF 

 

Experienced 

Nearly 

Experienced 

Neither 

Experienced Nor 

Nearly 

Experienced But 

Is A 

Hazard 

Not 

A 

Hazard 

 1. Electrical Shock ___ ___ ___ ___ 

 2. Burns ___ ___ ___ ___ 

 3. Cut or Abrasions ___ ___ ___ ___ 

 4. Extreme Brightness ___ ___ ___ ___ 

 5. Extreme Loudness ___ ___ ___ ___ 

 6. Noxious Fumes ___ ___ ___ ___ 

 7. Other (specify) ___ ___ ___ ___ 

8. Explanation of the above items which received checks in the 

 “Experienced”, “Nearly Experienced” and “Is a Hazard” categories. 

____________________________________________________________________________________________ 

____________________________________________________________________________________________ 

____________________________________________________________________________________________ 

9. Were any of the procedures you were required to perform during 

 emplacement or displacement of the system unsafe in that you risked 

 personal harm or damage to the facility by performing them? 

 ( ) Yes ( ) No 

 Explanation of "Yes" _________________________________________________________________________ 

_____________________________________________________________________________________________ 

_____________________________________________________________________________________________ 

 
160

INTENTIONALLY LEFT BLANK.
161

Appendix D. Human Factors Engineering Evaluation of Regency 

Net Equipment

 

This appendix appears in its original form, without editorial change. 
162

Regency 

Human Factors Engineering Evaluation of Regency Net Equipment 

Form 601-4-65 

Purpose: The items on this form ask you to evaluate various human factors 

aspects of the equipment you use. Given this and other information, 

corrective action can be taken to improve equipment design, operator training, 

task requirements, and task performance. Only Army Research Institute 

researchers will have access to your ratings. Therefore, please be honest and 

objective in your ratings. 

To start with: Enter you PIN number, and current rank in the spaces 

provided. Circle your current MOS. Your PIN number is required so that ARI 

research personnel may contact you if they need more information regarding 

your responses. 

1. PIN No. │___│___│___│ 2. Rank: │___│___│___│

3. Circle your current MOS: 31C 29S 52 29E Other_________ 

The Regency Net system is composed of many items of equipment, such as 

I/O Units, Vehicular Adapters, Receiver-Transmitters. Operation and 

maintenance of the equipment may require manual or electronic adjustment of 

different parts (e.g., display screens, touchpoints, connectors, cables, 

indicator lights) of these components. On each of the following pages an 

equipment component will be identified, and several human factors aspects of 

the component will be noted. Some of these human factors aspects may be 

unsatisfactory, needing correction or modification; some may be satisfactory 

as they are. Your task is to rate how satisfactory-unsatisfactory each Listed 

human factor is for the component named on each page. There are five rating 

categories available for your use. They have the following names and 

abbreviations: 

 RATING CATEGORIES ABBREVIATION NUMERICAL 

 VALUE 

 Very Unsatisfactory VU -2 

 Unsatisfactory U -1 

 Borderline B 0 

 Satisfactory S +1 

 Very Satisfactory VS +2 

 The rating scale below contains the abbreviations (to save space) of these 

five categories. The purpose of the numbers is to show that the ratings change from 

negative to positive, with a 0 under B (for Borderline) to show it is neither negative or 

positive. The underlines give you a place on which to mark your choice of the five 

ratings. 

 VU U B S VS

 -2 -1 0 +1 +2 

 

 __ __ __ __ __ (Check beneath the rating 

 you choose to give. 
163

RATINGS OF DISPLAY SCREEN FACTORS 

 Rating Categories VU U B S VS

 -2 -1 0 +1 +2 

VU = Very Unsatisfactory 

 U = Unsatisfactory 

 B = Borderline 

 S = Satisfactory 

VS = Very Satisfactory 

 VU U B S VS

 -2 -1 0 +1 +2 

4. Brightness............................................ __ __ __ __ __ 

5. Glare................................................... __ __ __ __ __ 

6. Flicker................................................. __ __ __ __ __ 

7. Letter clarity.......................................... __ __ __ __ __ 

8. Viewing distance.................................... __ __ __ __ __ 

9. Angle of view......................................... __ __ __ __ __ 

10. Location of display................................. __ __ __ __ __ 

 If any of the above human factors aspects reduced your task performance 

effectiveness, a brief comment about it and its effect would be appreciated. 

Please enter the item number in front of your comment. 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 
164

RATINGS OF TOUCHPOINT FACTORS 

 Rating Categories VU U B S VS

 -2 -1 0 +1 +2 

VU = Very Unsatisfactory 

 U = Unsatisfactory 

 B = Borderline 

 S = Satisfactory 

VS = Very Satisfactory 

 VU U B S VS

 -2 -1 0 +1 +2 

11. Size................................................... __ __ __ __ __ 

12. Shape................................................ __ __ __ __ __ 

13. Spacing.............................................. __ __ __ __ __ 

14. Label understandability........................... __ __ __ __ __ 

15. Label relevance.................................... __ __ __ __ __ 

16. Label location......................................... __ __ __ __ __ 

17. Brightness.............................................. __ __ __ __ __ 

 If any of the above human factors aspects reduced your task performance 

effectiveness, a brief comment about it and its effect would be appreciated. 

Please enter the item number in front of your comment. 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 
165

RATINGS OF INDICATOR LIGHT FACTORS 

 Rating Categories VU U B S VS

 -2 -1 0 +1 +2 

VU = Very Unsatisfactory 

 U = Unsatisfactory 

 B = Borderline 

 S = Satisfactory 

VS = Very Satisfactory 

 VU U B S VS

 -2 -1 0 +1 +2 

19. Brightness............................................ __ __ __ __ __ 

20. Glare................................................... __ __ __ __ __ 

21. Flicker................................................. __ __ __ __ __ 

22. Viewing distance.................................... __ __ __ __ __ 

23. Angle of view......................................... __ __ __ __ __ 

24. Label understandability............................ __ __ __ __ __ 

25. Label correctness................................. __ __ __ __ __ 

26. Label visibility...................................... __ __ __ __ __ 

27. Location of indicators............................ __ __ __ __ __ 

28. Provides needed information 

 in a timely manner................................ __ __ __ __ __ 

 If any of the above human factors aspects reduced your task performance 

effectiveness, a brief comment about it and its effect would be appreciated. 

Please enter the item number in front of your comment. 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 
166

RATINGS OF CONNECTOR FACTORS 

 Rating Categories VU U B S VS

 -2 -1 0 +1 +2 

VU = Very Unsatisfactory 

 U = Unsatisfactory 

 B = Borderline 

 S = Satisfactory 

VS = Very Satisfactory 

 VU U B S VS

 -2 -1 0 +1 +2 

29. Brightness............................................ __ __ __ __ __ 

30. Glare................................................... __ __ __ __ __ 

31. Flicker................................................. __ __ __ __ __ 

32. Letter clarity.......................................... __ __ __ __ __ 

33. Viewing distance.................................... __ __ __ __ __ 

34. Angle of view......................................... __ __ __ __ __ 

35. Location of display................................. __ __ __ __ __ 

36. Label understandability........................... __ __ __ __ __ 

37. Label location....................................... __ __ __ __ __ 

 If any of the above human factors aspects reduced your task performance 

effectiveness, a brief comment about it and its effect would be appreciated. 

Please enter the item number in front of your comment. 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 
167

RATINGS OF CABLE FACTORS 

 Rating Categories VU U B S VS

 -2 -1 0 +1 +2 

VU = Very Unsatisfactory 

 U = Unsatisfactory 

 B = Borderline 

 S = Satisfactory 

VS = Very Satisfactory 

 VU U B S VS

 -2 -1 0 +1 +2 

38. Coding on insulation 

 (should be about every 

 12 inches) ......................................... __ __ __ __ __ 

39. Routing (for easy of 

 inspection & maintenance)...................... __ __ __ __ __ 

40. Positioned to avoid damage 

 by tools, hands, feet............................... __ __ __ __ __ 

41. Safety (protection from 

 sharp edges)....................................... __ __ __ __ __ 

42. Mating information (labeled 

 to indicate where matings 

 are to occur).................................... __ __ __ __ __ 

 If any of the above human factors aspects reduced your task performance 

effectiveness, a brief comment about it and its effect would be appreciated. 

Please enter the item number in front of your comment. 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 
168

RATINGS OF DISPLAY SCREEN FACTORS 

 Rating Categories VU U B S VS

 -2 -1 0 +1 +2 

VU = Very Unsatisfactory 

 U = Unsatisfactory 

 B = Borderline 

 S = Satisfactory 

VS = Very Satisfactory 

 VU U B S VS

 -2 -1 0 +1 +2 

43. Frequency (too high, too low).................... __ __ __ __ __ 

44. Detectability.......................................... __ __ __ __ __ 

45. Intensity.............................................. __ __ __ __ __ 

46. Duration.......................................... __ __ __ __ __ 

47. Ease of volume adjustment..................... __ __ __ __ __ 

48. Amount of volume 

 adjustment available............................. __ __ __ __ __ 

 If any of the above human factors aspects reduced your task performance 

effectiveness, a brief comment about it and its effect would be appreciated. 

Please enter the item number in front of your comment. 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 
169

RATINGS OF PUSHBUTTON AND SWITCH FACTORS 

 Rating Categories VU U B S VS

 -2 -1 0 +1 +2 

VU = Very Unsatisfactory 

 U = Unsatisfactory 

 B = Borderline 

 S = Satisfactory 

VS = Very Satisfactory 

 VU U B S VS

 -2 -1 0 +1 +2 

49. Shape (concave - to avoid 

 finger slipping).................................... __ __ __ __ __ 

50. System responsiveness 

 (evidence of system 

 response when pushed)....................... __ __ __ __ __ 

51. Size................................................... __ __ __ __ __ 

52. Resistance (too hard, 

 too sensitive)....................................... __ __ __ __ __ 

53. Amount of displacement 

 required.............................................. __ __ __ __ __ 

54. Label 

 understandability................................... __ __ __ __ __ 

 If any of the above human factors aspects reduced your task performance 

effectiveness, a brief comment about it and its effect would be appreciated. 

Please enter the item number in front of your comment. 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 
170

RATINGS OF TOGGLE SWITCH FACTORS 

 Rating Categories VU U B S VS

 -2 -1 0 +1 +2 

VU = Very Unsatisfactory 

 U = Unsatisfactory 

 B = Borderline 

 S = Satisfactory 

VS = Very Satisfactory 

 VU U B S VS

 -2 -1 0 +1 +2 

55. Size................................................... __ __ __ __ __ 

56. Accidental change of 

 switch position?.................................... __ __ __ __ __ 

57. Distance between 

 on and off positions............................... __ __ __ __ __ 

58. System responsiveness 

 (evidence of system 

 response when pushed)........................... __ __ __ __ __ 

59. Direction of movement 

 (vertical with "down" 

 to turn off)............................................... __ __ __ __ __ 

60. Labeling (for each 

 position)................................................. __ __ __ __ __ 

 If any of the above human factors aspects reduced your task performance 

effectiveness, a brief comment about it and its effect would be appreciated. 

Please enter the item number in front of your comment. 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 
171

RATINGS OF FUSE FACTORS 

 Rating Categories VU U B S VS

 -2 -1 0 +1 +2 

VU = Very Unsatisfactory 

 U = Unsatisfactory 

 B = Borderline 

 S = Satisfactory 

VS = Very Satisfactory 

 VU U B S VS

 -2 -1 0 +1 +2 

61. Positive indication of 

 open circuit......................................... __ __ __ __ __ 

62. Availability of special 

 tools to remove other 

 components....................................... __ __ __ __ __ 

63. Identification label is 

 on equipment........................................ __ __ __ __ __ 

64. Rating is marked on fuse 

 holder................................................. __ __ __ __ __ 

65. Label legibility....................................... __ __ __ __ __ 

66. Ground connection................................ __ __ __ __ __ 

67. Fan operation....................................... __ __ __ __ __ 

 If any of the above human factors aspects reduced your task performance 

effectiveness, a brief comment about it and its effect would be appreciated. 

Please enter the item number in front of your comment. 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 
172

RATINGS OF KEYPAD FACTORS 

 Rating Categories VU U B S VS

 -2 -1 0 +1 +2 

VU = Very Unsatisfactory 

 U = Unsatisfactory 

 B = Borderline 

 S = Satisfactory 

VS = Very Satisfactory 

 VU U B S VS

 -2 -1 0 +1 +2 

68. Size..................................................... __ __ __ __ __ 

69. Shape................................................ __ __ __ __ __ 

70. Spacing of keys..................................... __ __ __ __ __ 

71. Resistance (too easy or 

 too hard to push).................................... __ __ __ __ __ 

72. Label correctness.................................... __ __ __ __ __ 

73. Label understandability............................. __ __ __ __ __ 

74. Label location....................................... __ __ __ __ __ 

75. Brightness........................................... __ __ __ __ __ 

76. System responsiveness 

 to key touching..................................... __ __ __ __ __ 

77. Acceptability of keyboard........................ __ __ __ __ __ 

 If any of the above human factors aspects reduced your task performance 

effectiveness, a brief comment about it and its effect would be appreciated. 

Please enter the item number in front of your comment. 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 
173

RATINGS OF HANDSET AUDIO FACTORS 

 Rating Categories VU U B S VS

 -2 -1 0 +1 +2 

VU = Very Unsatisfactory 

 U = Unsatisfactory 

 B = Borderline 

 S = Satisfactory 

VS = Very Satisfactory 

 VU U B S VS

 -2 -1 0 +1 +2 

78. Ease of volume adjustment....................... __ __ __ __ __ 

79. Amount of volume 

 adjustment............................................. __ __ __ __ __ 

 If any of the above human factors aspects reduced your task performance 

effectiveness, a brief comment about it and its effect would be appreciated. 

Please enter the item number in front of your comment. 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 
174

RATINGS OF ROTARY SWITCH FACTORS 

 Rating Categories VU U B S VS

 -2 -1 0 +1 +2 

VU = Very Unsatisfactory 

 U = Unsatisfactory 

 B = Borderline 

 S = Satisfactory 

VS = Very Satisfactory 

 VU U B S VS

 -2 -1 0 +1 +2 

80. Shapes................................................ __ __ __ __ __ 

81. Position (not opposite 

 each other).......................................... __ __ __ __ __ 

82. Resistance to turning ............................. __ __ __ __ __ 

83. Pointer visibility....................................... __ __ __ __ __ 

84. Parallax (pointer too far 

 from scale?)......................................... __ __ __ __ __ 

85. Distance between index 

 marks on scale...................................... __ __ __ __ __ 

86. System responsiveness 

 (to movement of pointer).......................... __ __ __ __ __ 

 If any of the above human factors aspects reduced your task performance 

effectiveness, a brief comment about it and its effect would be appreciated. 

Please enter the item number in front of your comment. 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 
175

RATINGS OF VOLTAGE METER FACTORS 

 Rating Categories VU U B S VS

 -2 -1 0 +1 +2 

VU = Very Unsatisfactory 

 U = Unsatisfactory 

 B = Borderline 

 S = Satisfactory 

VS = Very Satisfactory 

 VU U B S VS

 -2 -1 0 +1 +2 

87. Readable (in all ambient 

 lighting conditions).................................. __ __ __ __ __ 

88. Glare................................................... __ __ __ __ __ 

89. Legibility of numbers............................... __ __ __ __ __ 

90. Viewing distance.................................... __ __ __ __ __ 

91. Label location......................................... __ __ __ __ __ 

92. Location of meter................................... __ __ __ __ __ 

93. Adjustability (for visual 

 access)................................................ __ __ __ __ __ 

 If any of the above human factors aspects reduced your task performance 

effectiveness, a brief comment about it and its effect would be appreciated. 

Please enter the item number in front of your comment. 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 
176

RATINGS OF SLIDE SWITCH FACTORS 

 Rating Categories VU U B S VS

 -2 -1 0 +1 +2 

VU = Very Unsatisfactory 

 U = Unsatisfactory 

 B = Borderline 

 S = Satisfactory 

VS = Very Satisfactory 

 VU U B S VS

 -2 -1 0 +1 +2 

94. Protection against 

 accidental activation................................ __ __ __ __ __ 

95. Size ..................................................... __ __ __ __ __ 

96. Resistance pattern when 

 being turned.......................................... __ __ __ __ __ 

97. Distance between positions 

 (RESET/OFF)........................................ __ __ __ __ __ 

98. System responsiveness 

 (to movement of slide 

 switch).................................................. __ __ __ __ __ 

99. Orientation (vertical 

 motion downward to 

 turn off)................................................. __ __ __ __ __ 

 If any of the above human factors aspects reduced your task performance 

effectiveness, a brief comment about it and its effect would be appreciated. 

Please enter the item number in front of your comment. 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 

___________________________________________________________________ 

 
177

Appendix E. Resource Material for Health Hazards, System Safety, and 

Risk Assessment Matrix 

 

This appendix appears in its original form, without editorial change. 
178

RESOURCE MATERIAL FOR HEALTH HAZARDS, SYSTEM SAFETY AND RISK 

ASSESSMENT MATRIX 

 Much of the material contained in this section has been abstracted from a document 

entitled: CLASSIFICATION OF DEFICIENCIES AND SHORTCOMINGS. This material 

represents an abstract of selected sections that have been found directly relevant to implementing 

MANPRINT assessment in these domains. In this abstraction process, material is abstracted 

from the whole text and is sometimes quoted in total; other times it is paraphrased.4

 

1. Purpose and Scope. This material describes criteria intended to provide consistent guidance 

for classification of appropriate material test incidents and test results as deficiencies and 

shortcomings and is used to address problems identified by the MANPRINT Analysts that are 

generally categorized as falling within the Health Hazards and System Safety domains. While 

other organizational entities have primary responsibility for certification within these domains, 

the MANPRINT Analyst reports problems falling within these domains that he observes and/or 

have been reported to him by test participants. As such, the basic data analyzed by the 

MANPRINT Analyst are Test Incident Reports (TIRs). Occasionally those participants are 

Subject Matter Experts (SME); most often they are soldiers who are expected to operate and 

maintain the system under test once it is fielded. All guidance concerning classification of test 

incidents is intended for use in classifying deficiencies, shortcomings, suggested improvements, 

and other incidents. When SMEs are used this intent is more nearly met than when 

representative test participants are providing their opinions. As such opinions and judgments are 

combined to provide frequency of the reported problems. Categorization of those problems is 

used as an indicant of a problem that may require further study by the Training System Manager 

prior to recommending changes in the material system or operating procedures. 

2. TECHNICAL PRESENTATION 

 a. Deficiency. 

1) Definition: A deficiency has been defined as a defect or malfunction 

discovered during the life cycle of an equipment that constitutes a safety 

hazard to personnel that will result in serious damage to the equipment if 

operation is continued or indicates improper design or other cause of failure of 

an item or part which seriously impairs the equipment's operational capability. 

A deficiency normally disables or immobilizes the equipment; and if 

occurring during test phases, will serve as a bar to type classification action. 

 

4 Effort has been to focus on content and usability of the material, but generally there has been no conscious effort 

made to attribute specific authorship when the material was quoted verbatim. There is no claim or representation to 

indicate that the author of this Tutorial document claims credit for the specific wording when quotes are used, but 

not credited. This liberty has been taken as an expedient to pull together a sizeable body of knowledge in a short 

time. In those cases, the author of this document requests forgiveness by the true authors. Material contained herein 

has been extracted from a now obsolete document: Test Operations Procedures 1-1-012, CLASSIFICATION OF 

DEFICIENCIES AND SHORTCOMINGS, 1 April 1979.
179

 2) Classification: In analysis of test results great care must be taken to insure 

proper classification of a test incident as a deficiency or a shortcoming. The use of judgment, 

both technical and military, is necessary together with the use of regulating criteria in the 

analysis of test incidents before classifying them. To adequately understand the fine difference 

dividing a deficiency from a shortcoming it is necessary to examine the definitions in great 

detail. In order for a test incident to be considered a deficiency it must (1) be a characteristic 

which causes the failure, not the failure itself, (2) materially and seriously degrade the 

operational capability of the item, (3) not be the result of an isolate failure, (4) apply to the 

population from which the sample was drawn, and (5) be stated with reasonable certainty that it 

is characteristic of the items under test. 

 b. Shortcoming.

 Definition: A shortcoming is an imperfection or malfunction occurring during the 

life cycle of equipment, which should be reported and which must be corrected to increase 

efficiency and to render the equipment completely serviceable. I will not cause an immediate 

breakdown, jeopardize safe operation, or materially reduce the usability of the material or end 

product. If occurring during test phases the shortcoming should be corrected if it can be done 

without unduly complicating an item or inducing another undesirable characteristic such as 

increased cost, weight, etc. 

 c. Suggested Improvements.

 Definition: A suggested improvement is an increase in quality or performance 

which is desirable but not imperative. 

 d. Hazard Level Categories. MIL-STD-882 requires that safety hazards be categorized by 

both severity and probability. When these ratings are obtained they are generally asked for 

through separate instruments to increase the likelihood that the judgments are independent. It is 

not appropriate to classify all catastrophic and critical hazards as deficiencies and all marginal 

hazards as shortcomings. The proper classification procedures are shown in Figure 2. The 

following definitions apply to the terms used in that figure. 

 1) Hazard Severity. A qualitative assessment of the worst potential 

consequences, defined by the degree of injury, occupational illness, property damage, or 

equipment damage that could ultimately occur. 

 2) Hazard Probability. The likelihood, expressed in quantitative or qualitative 

terms, that a hazard will occur. 
180

FREQUENT 

REASONABLY 

PROBABLE OCCASIONAL REMOTE 

EXTREMEL

IMPROBAB

 SPECIFIC 

INDIVIDUAL 

ITEM 

Likely to occur  frequently 

Will occur several 

times in life of 

item 

Likely to occur  sometime in the  life of item 

So unlikely, can be assumed that this hazard will not be experienced 

Probability of occurrence can't distinguished from zero 

FLEET OR 

INVENTORY 

Continuously experienced 

Will occur 

frequently 

Will occur 

 several times 

Unlikely to occur, but possible 

So unlikely, can assumed that this hazard will be experienced 

 A B C D E 

HAZARD SEVERITY 

CATASTROPHICMay cause death or 

system loss 

I DEFICIENCY DEFICIENCY DEFICIENCY DEFICIENCY 

SUGGESTE

IMPROVEME

OR ACCEPTA

CRITICALMay cause severe 

Injury or illness, or 

major system 

damage 

II DEFICIENCY DEFICIENCY DEFICIENCY SHORTCOMING 

SUGGESTE

IMPROVEME

OR 

ACCEPTAB

MARGINALMay cause minor 

 injury or illness, or 

minor system 

damage 

III DEFICIENCY SHORTCOMING SHORTCOMING 

SUGGESTED 

IMPROVEMENT 

SUGGESTE

IMPROVEME

OR ACCEPTA

NEGLIGIBLEWill not result in 

injury or illness, or 

system damage 

IV SHORTCOMING 

SUGGESTED 

IMPROVEMENT 

SUGGESTED 

IMPROVEMENT 

SUGGESTED 

IMPROVEMENT 

OR 

ACCEPTABLE 

SUGGESTE

IMPROVEME

OR ACCEPTA

Figure 1 Hazard Probability vs. Hazard Severity: Risk Assessment Matrix 
181

Appendix F. Maneuver Control System (MCS) Training Questionnaire

 

This appendix appears in its original form, without editorial change. 
182

MCS/P IOT&E 

TRAINING QUESTIONNAIRE

June 24 1996 

Privacy Act Statement 

Authority; 5 USC § 301, Authority for the Secretary of the Army to Issue Army Regulations, AR 73-1, Test and Evaluation Policy. 

Principal Purpose: The data to be collected with this form are to be used for research and evaluation purposes only. 

Routine Uses: This is an experimental data collection questionnaire developed by the Test and Experimentation Command pursuant to its 

research and testing mission as prescribed in AR 73-1. When identifier (name and social security number) are requested they are to be used for 

administrative and statistical control purposes only. Full confidentiality of the responses will be maintained in the processing of these data. 

Disclosure: Completion of this questionnaire is voluntary. You are encouraged to provide complete and accurate information in the interests of 

research and testing, but there will be no effect on individuals for not providing any part of the information. 

1.) Date:_____________________ 

2.) Name:____________________│____________________ 

 (last name, first name) 

3.) Questionnaire Serial #: TR __ __ __ 

4.) Last four digits of your social security number: __ __ __ __ 

5.) Duty position (mark one □): □ operator □ supervisor □ staff □ commander □ other 

6.) Training Dates: Start (YYMMDD) __ __ __ __ __ __ End (YYMMDD) __ __ __ __ __ __ 

Instructions 

The purpose of this questionnaire is to get your opinions about MCS/P. The questionnaire is important because your 

answers will help us judge how well MCS/P has been built for soldiers. Your answers will not be given to or shown 

to anyone except those who are evaluating MCS/P for the Army. (For example, none of your information will be 

given to your chain of command or put in your personnel file) Your answers will be treated confidentially. Please 

fill out the questionnaire carefully. If you need additional space to answer a question, indicate by an arrow (→) and 

continue on the back of the page. Be sure to number the item on the back of the page. If you have any questions 

concerning this questionnaire, please contact a TEXCOM representative for help. Thank you for your help.

1 of 7 
183

PART 1: Training Evaluation 

7.) Which type of training are you evaluating? (mark one □ ) 

□ Staff User (SU) □ System Manager (SM) □ System Maintainer □ Senior Staff (SS) □ Executive (EX) 

8.) The amount of time spent on hands-on training was (mark one □): 

□ very adequate □ adequate □ borderline □ inadequate □ very inadequate 

9. The amount of time spent you had to ask questions was (mark one □): 

□ very adequate □ adequate □ borderline □ inadequate □ very inadequate 

10.) Did this training train you to a level that allowed you to operate the equipment without help from someone 

else? (mark one □ ): 

□ yes □ no → please explain: 

_____________________________________________________________ 

 

_____________________________________________________________ 

 

_____________________________________________________________ 

11.) The use of training support materials (TMs, handouts, videos, Vu-graph slides) was (mark one □): 

□ very adequate □ adequate □ borderline □ inadequate □ very inadequate 

12.) The opportunities to practice skills with other operators was (mark one □): 

□ very adequate □ adequate □ borderline □ inadequate □ very inadequate 

13.) The opportunities to get help from instructors was (mark one □): 

□ very adequate □ adequate □ borderline □ inadequate □ very inadequate 

14.) The content of instructor handouts was (mark one □):

□ very appropriate □ appropriate □ borderline □ inappropriate □ very inappropriate 

2 of 7 
184

15. The format of instructor handouts was (mark one □): 

□ very appropriate □ appropriate □ borderline □ inappropriate □ very inappropriate 

16. The content of Vu-graph slides (overhead slides) was (mark one □): 

□ very appropriate □ appropriate □ borderline □ inappropriate □ very inappropriate 

17.) The format of Vu-graph slides (overhead slides) was (mark one □): 

□ very appropriate □ appropriate □ borderline □ inappropriate □ very inappropriate 

18.) Do the guide sheets for practical exercises make sense? (mark one □): 

□ yes □ no → please explain: ________________________________________________ 

 ________________________________________________ 

 ________________________________________________ 

 ________________________________________________ 

19.) Are critical (primary) tasks clearly identified (mark one □): 

□ yes □ no → please explain: ________________________________________________ 

 ________________________________________________ 

 ________________________________________________ 

 ________________________________________________ 

20.) Are the training goals of this training clearly stated? (mark one □): 

□ yes □ no → please explain: ________________________________________________ 

 ________________________________________________ 

 ________________________________________________ 

 ________________________________________________ 

21.) The standards used to evaluate your progress were (mark one □): 

□ very appropriate □ appropriate □ borderline □ inappropriate □ very inappropriate 

3 of 7 
185

22.) Did you have a chance to practice new skills after they were taught (practice did not have to wait until many 

new skills were presented? (mark one □): 

□ yes □ no → please explain: ________________________________________________ 

 ________________________________________________ 

 ________________________________________________ 

 ________________________________________________ 

23.) Training time was used (mark one □): 

□ very effectively □ effectively □ borderline □ ineffectively □ very ineffectively 

24.) The instruction on the use of equipment TMs was (mark one □): 

□ very adequate □ adequate □ borderline □ inadequate □ very inadequate 

25.) System safety issues addressed in class were (mark one □): 

□ very adequate □ adequate □ borderline □ inadequate □ very inadequate 

26.) Is the software version trained the same version installed on the equipment used in class? 

(mark one □): 

□ yes □ no → please explain: ________________________________________________ 

 ________________________________________________ 

 ________________________________________________ 

 ________________________________________________ 

27.) The pace at which new material is presented in class is (mark one □): 

□ very appropriate □ appropriate □ borderline □ inappropriate □ very inappropriate 

28.) Were you allowed to practice all new or updated tasks in class? (mark one □): 

□ yes □ no → please explain: ________________________________________________ 

 ________________________________________________ 

 ________________________________________________ 

 ________________________________________________ 

4 of 7 
186

29.) When compared to the length of class time, the number of new tasks/procedures was (mark one □): 

□ very adequate □ adequate □ borderline □ inadequate □ very inadequate 

30.) Was refresher training on old tasks and procedures provided? (mark one □): 

□ yes □ no → please explain: __________________________________________________ 

 ________________________________________________ 

 ________________________________________________ 

 ________________________________________________ 

31.) After you completed the training, do you have the ability to perform all critical tasks and procedures? 

(mark one □): 

□ yes □ no → please explain: __________________________________________________ 

 ________________________________________________ 

 ________________________________________________ 

 ________________________________________________ 

PART II: Training Devices 

Training Device 

Technique or Topic 

Used During Training? Time Spent

NO YES Too 

Much 

About 

Right 

Too 

Little 

32.) Lecture and 

 Discussion 

□ □ →→→ □ □ □

33.) Hands-on 

 Training 

□ □→→ → □ □ □

34.) Refresher 

 Training on 

 Old Tasks 

□

 □ →→→ □ □ □

35.) Practical 

 Exercises 

□ □→→→ □ □ □

36.) Instruction of 

 Critical Tasks 

□ □ →→→ □ □ □

37.) Discussion of 

 Safety Issues 

□ □ →→ → □ □ □

38.) Actual Equip- 

 ment to be 

 Operated in 

 Unit 

□ □→→→ □ □ □

 5 of 7 
187

PART II: Training Devices (continued) 

Training Device 

Technique or Topic 

Used During Training? Time Spent

NO YES Too 

Much 

About 

Right 

Too 

Little 

39.) Video Tapes □ □ →→→ □ □ □

40.) Technical 

 Manuals 

□ □ →→→ □ □ □

□ □ →→→ □ □ □

42.) Vu-graphs 

 (overhead slides) 

□ □ →→→ □ □ □

43.) Computer Screen 

 Projector 

□ □ →→→ □ □ □

44.) Actual Commun- 

 ications Network 

□ □ →→→ □ □ □

45.) Total Amount of 

 Training Time →→→ □ □ □

PART III: MCS/P Tasks

KEY: M=Need Much More, SM=Need Some More, R=About Right, SL=Need Some Less, L=Need Much Less 

Task or Operation 

Task 

Trained

Training Time/

Task Coverage

No Yes M SM R SL L 

46.) Perform MCS/P 

 and SICPS 

 Maintenance 

□ □ → □ □ □ □ □

47) Prepare MCS/P 

 for Operations 

□ □ → □ □ □ □ □

48.) Perform PMCS 

 on the MCS/P 

 Equipment 

□ □ → □ □ □ □ □

49.) Prepare MCS/P 

 for Movement □ □ → □ □ □ □ □

50.) Troubleshoot/ 

 Fault Isolate the 

 MCS/P 

 Equipment 

□ □ → □ □ □ □ □

51.) Employ a SICPS □ □ → □ □ □ □ □

 6 of 7 
188

PART III: MCS/P Tasks (continued) 

KEY: M=Need Much More, SM=Need Some More, R=About Right, SL=Need Some Less, L=Need Much Less 

Task 

Trained

Training Time/

Task Coverage

Task or Operation No Yes M SM R SL L 

52.) Perform PMCS on the 

 SICPS Equipment □ □ → □ □ □ □ □

53.) Troubleshoot/ Fault 

 Isolate the SICPS 

 Equipment 

□ □ → □ □ □ □ □

54.) Prepare the SICPS for 

 Movement □ □ → □ □ □ □ □

55.) Troubleshoot/Fault 

 Isolate and Restore 

 the LAN 

□ □ → □ □ □ □ □

56.) Perform Information 

 Security Functions □ □ → □ □ □ □ □

57.) Perform System 

 Administration Functions □ □ → □ □ □ □ □

58.) Operate UTO Software □ □ → □ □ □ □ □

59.) Operate Reports 

 Software □ □ → □ □ □ □ □

60.) Operate Message 

 Software 

 

□→ □ □ □ □ □

61.) Perform System Manager 

 Functions 

□ 

□ →

□ □ □ □

62.) Operate OPORD 

 Software □ □ → □ □ □ □ □

63.) Operate Maps and 

 Overlays Software □ □ → □ □ □ □ □

64.) Perform Briefing System 

 Functions □ □ → □ □ □ □ □

65.) Perform Telestrator 

 Functions □ □ → □ □ □ □ □

66.) Plan for CONOPS □ □→ □ □ □ □ □

67.) Plan the MCS/P Network 

 Architecture □ □ → □ □ □ □ □

68.) TEST SCORES (to be provided by TEXCOM): 

 SM PT #1:__ __ __ SM PT #2: __ __ __ 

 SS WE: __ __ __ SS PE: __ __ __ 

7 of 7 
189

Appendix G. Maneuver Control System Workload Questionnaire

 

This appendix appears in its original form, without editorial change. 
190

MCS/P IOT&E 

WORKLOAD QUESTIONNAIRE

June 24 1996 

Privacy Act Statement 

Authority; 5 USC § 301, Authority for the Secretary of the Army to Issue Army Regulations, AR 73-1, Test and Evaluation Policy. 

Principal Purpose: The data to be collected with this form are to be used for research and evaluation purposes only. 

Routine Uses: This is an experimental data collection questionnaire developed by the Test and Experimentation Command pursuant to its 

research and testing mission as prescribed in AR 73-1. When identifier (name and social security number) are requested they are to be used for 

administrative and statistical control purposes only. Full confidentiality of the responses will be maintained in the processing of these data. 

Disclosure: Completion of this questionnaire is voluntary. You are encouraged to provide complete and accurate information in the interests of 

research and testing, but there will be no effect on individuals for not providing any part of the information. 

1.) Date:_____________________ 

2.) Rank: __ __ __ 

3.) Name:____________________│____________________ 

 (last name, first name) 

4.) Questionnaire Serial #: WL __ __ __ 

5.) Last four digits of your social security number: __ __ __ __ 

6.) Duty position (mark one □): □ staff user □ system manager □ staff □ commander 

Instructions: Using the definitions provided below, darken the appropriate circle on the scale 

For each aspect for each task listed. Please darken only in one of the circles provided. The 

Results of this questionnaire will be used to help the U.S. Army improve MCS/P for your use. 

Thanks for the help.

Definitions of the six aspects you will be using with rating scales: 

Aspect Description 

Mental Demand............................... How mentally demanding was the task? 

Physical Demand............................. How physically demanding was the task? 

Temporal Demand........................... How hurried or rushed was the pace of the task? 

Performance................................... How successful were you in accomplishing what 

you were asked to do? 

Effort............................................. How hard did you have to work to accomplish 

your level of performance? 

Frustration...................................... How insecure, discouraged, irritated, and annoyed 

 were you? 

 

Page 1 of 12 
191

1. Perform MCS/P and SICPS Maintenance. 

Mental Demand  Very Very 

 Low High 

Physical Demand  Very Very 

 Low High 

Temporal Demand  Very Very 

 Low High 

Performance  Very Very 

 Low High 

Effort  Very Very 

 Low High 

Frustration  Very Very 

 Low High 

__________________________________________ 

2. Prepare MCS/P for Operations. 

Mental Demand  Very Very 

 Low High 

Physical Demand  Very Very 

 Low High 

Temporal Demand  Very Very 

 Low High 

Performance  Very Very 

 Low High 

Effort  Very Very 

 Low High 

Frustration  Very Very 

 Low High 

 

Page 2 of 12 
192

3. Perform PMCS on the MCS/P Equipment. 

Mental Demand  Very Very 

 Low High 

Physical Demand  Very Very 

 Low High 

Temporal Demand  Very Very 

 Low High 

Performance  Very Very 

 Low High 

Effort  Very Very 

 Low High 

Frustration  Very Very 

 Low High 

__________________________________________ 

4. Prepare MCS/P for Movement. 

Mental Demand  Very Very 

 Low High 

Physical Demand  Very Very 

 Low High 

Temporal Demand  Very Very 

 Low High 

Performance  Very Very 

 Low High 

Effort  Very Very 

 Low High 

Frustration  Very Very 

 Low High 

 

Page 3 of 12 
193

5. Troubleshoot/Fault Isolate the MCS/P Equipment. 

Mental Demand  Very Very 

 Low High 

Physical Demand  Very Very 

 Low High 

Temporal Demand  Very Very 

 Low High 

Performance  Very Very 

 Low High 

Effort  Very Very 

 Low High 

Frustration  Very Very 

 Low High 

__________________________________________ 

6. Employ a SICPS 

Mental Demand  Very Very 

 Low High 

Physical Demand  Very Very 

 Low High 

Temporal Demand  Very Very 

 Low High 

Performance  Very Very 

 Low High 

Effort  Very Very 

 Low High 

Frustration  Very Very 

 Low High 

 

Page 4 of 12 
194

7. Perform PMCS on the SICPS Equipment. 

Mental Demand  Very Very 

 Low High 

Physical Demand  Very Very 

 Low High 

Temporal Demand  Very Very 

 Low High 

Performance  Very Very 

 Low High 

Effort  Very Very 

 Low High 

Frustration  Very Very 

 Low High 

__________________________________________ 

8. Troubleshoot/Fault Isolate SICPS Equipment.. 

Mental Demand  Very Very 

 Low High 

Physical Demand  Very Very 

 Low High 

Temporal Demand  Very Very 

 Low High 

Performance  Very Very 

 Low High 

Effort  Very Very 

 Low High 

Frustration  Very Very 

 Low High 

 

Page 5 of 12 
195

9. Prepare the SICPS for Movement. 

Mental Demand  Very Very 

 Low High 

Physical Demand  Very Very 

 Low High 

Temporal Demand  Very Very 

 Low High 

Performance  Very Very 

 Low High 

Effort  Very Very 

 Low High 

Frustration  Very Very 

 Low High 

__________________________________________ 

10. Troubleshoot/Fault Isolate and Restore the LAN. 

Mental Demand  Very Very 

 Low High 

Physical Demand  Very Very 

 Low High 

Temporal Demand  Very Very 

 Low High 

Performance  Very Very 

 Low High 

Effort  Very Very 

 Low High 

Frustration  Very Very 

 Low High 

 

Page 6 of 12 
196

11. Perform Information Security Functions. 

Mental Demand  Very Very 

 Low High 

Physical Demand  Very Very 

 Low High 

Temporal Demand  Very Very 

 Low High 

Performance  Very Very 

 Low High 

Effort  Very Very 

 Low High 

Frustration  Very Very 

 Low High 

__________________________________________ 

12. Perform System Administration Functions. 

Mental Demand  Very Very 

 Low High 

Physical Demand  Very Very 

 Low High 

Temporal Demand  Very Very 

 Low High 

Performance  Very Very 

 Low High 

Effort  Very Very 

 Low High 

Frustration  Very Very 

 Low High 

 

Page 7 of 12 
197

13. Operate UTO Software. 

Mental Demand  Very Very 

 Low High 

Physical Demand  Very Very 

 Low High 

Temporal Demand  Very Very 

 Low High 

Performance  Very Very 

 Low High 

Effort  Very Very 

 Low High 

Frustration  Very Very 

 Low High 

__________________________________________ 

14. Operate Reports Software. 

Mental Demand  Very Very 

 Low High 

Physical Demand  Very Very 

 Low High 

Temporal Demand  Very Very 

 Low High 

Performance  Very Very 

 Low High 

Effort  Very Very 

 Low High 

Frustration  Very Very 

 Low High 

 

Page 8 of 12 
198

15. Operate Message Software. 

Mental Demand  Very Very 

 Low High 

Physical Demand  Very Very 

 Low High 

Temporal Demand  Very Very 

 Low High 

Performance  Very Very 

 Low High 

Effort  Very Very 

 Low High 

Frustration  Very Very 

 Low High 

__________________________________________ 

16. Perform System Manager Functions. 

Mental Demand  Very Very 

 Low High 

Physical Demand  Very Very 

 Low High 

Temporal Demand  Very Very 

 Low High 

Performance  Very Very 

 Low High 

Effort  Very Very 

 Low High 

Frustration  Very Very 

 Low High 

 

Page 9 of 12 
199

17. Operate OPLAN/OPORD Software. 

Mental Demand  Very Very 

 Low High 

Physical Demand  Very Very 

 Low High 

Temporal Demand  Very Very 

 Low High 

Performance  Very Very 

 Low High 

Effort  Very Very 

 Low High 

Frustration  Very Very 

 Low High 

__________________________________________ 

18. Plan the MCS/P Network Architecture. 

Mental Demand  Very Very 

 Low High 

Physical Demand  Very Very 

 Low High 

Temporal Demand  Very Very 

 Low High 

Performance  Very Very 

 Low High 

Effort  Very Very 

 Low High 

Frustration  Very Very 

 Low High 

 

Page 10 of 12 
200

19. Plan for CONOPS. 

Mental Demand  Very Very 

 Low High 

Physical Demand  Very Very 

 Low High 

Temporal Demand  Very Very 

 Low High 

Performance  Very Very 

 Low High 

Effort  Very Very 

 Low High 

Frustration  Very Very 

 Low High 

__________________________________________ 

20. Execute CONOPS. 

Mental Demand  Very Very 

 Low High 

Physical Demand  Very Very 

 Low High 

Temporal Demand  Very Very 

 Low High 

Performance  Very Very 

 Low High 

Effort  Very Very 

 Low High 

Frustration  Very Very 

 Low High 

 

Page 11 of 12 
201

21. Perform Briefing System Functions. 

Mental Demand  Very Very 

 Low High 

Physical Demand  Very Very 

 Low High 

Temporal Demand  Very Very 

 Low High 

Performance  Very Very 

 Low High 

Effort  Very Very 

 Low High 

Frustration  Very Very 

 Low High 

__________________________________________ 

22. Perform Telestrator Functions. 

Mental Demand  Very Very 

 Low High 

Physical Demand  Very Very 

 Low High 

Temporal Demand  Very Very 

 Low High 

Performance  Very Very 

 Low High 

Effort  Very Very 

 Low High 

Frustration  Very Very 

 Low High 

 

Page 12 of 12 
202

INTENTIONALLY LEFT BLANK.
203

Appendix H. MANPRINT Survey/Interview Questions for All-Source 

Analysis System (ASAS) Block II Initial Operational Test 

and Evaluation (IOTE)

 

This appendix appears in its original form, without editorial change. 
204

Analysis and Control Element (ACE) IOTE 

MANPRINT SURVEY/INTERVIEW QUESTIONS 

Instructions: 

1. Please check the Functional Identities on which you have been serving as an Operator and indicate your Echelon

(e.g., Btn, Bde, Div) of operation. It is understood that some soldiers will be operating in more than one

functionality.

BIO/BACKGROUND/POSITION

Name ____________________ PIN# Date ______________

(Please Print) (Last 4 SSN) MM/DD/YY

Functional Identities (ACE Components) on which you will be serving as a Test Participant Operator (please check):

__ ISS

__CCS

__ Shared SS

__ COMINT

__ IMINT

__ RIS/ELINT

__ SIGINT/MASINT

__ CI/HUMINT

__ OSINT

__ AS

__SIT/IPB/DB

__ TGTDEV

__ CM/ISR

__Asset Mgmnt

__ Trusted Suite (TWS)

Echelon (Btn, Bde, Div)_____________

2. This instrument contains a series of Statements. Statements 1-9 focus on Equipment (items a-n) within the ACE. 

Statements 10-18 focus on Functions (items a1-t1) of the ACE. 

3. Statements differ only in the MANPRINT domain to which they refer. The domain is underlined in the stem of 

each statement. 

4. As you insert each item within each stem to form a statement, ask yourself whether this is true (Y) or false (N) 

and record the answer in the column to the left of the item. Put Not Applicable (N/A) if you did not operate this 

piece of Equipment (or perform this Function) in the ACE. 

5. For those cases where you recorded: true (Y), please make a Rating using the scale shown at the top of each page 

to record your amount of agreement with the Statement. Ratings should be recorded in the right hand column after 

each item. 

6. IN CASES WHERE YOU INDICATE TRUE (Y), AN EXPLANATION "WHY" IS NEEDED. PLEASE 

WRITE YOUR RESPONSE ON THE BACK OF THIS FORM AND INDICATE FOR WHICH QUESTION YOU 

ARE MAKING COMMENT. 
1 = Completely Disagree 3 = Generally Disagree 5 = Strongly Agree 9 = Unknown 

2 = Strongly Disagree 4 = Generally Agree 6 = Completely Agree 

205

Equipment Focus: Health Hazards

Y/N Rating

 1. Performing tasks or operations with ___________ jeopardized my 

 safety or that of another Team member. 

_____ a. TROJAN special purpose integrated remote intelligence 

 terminal (SPIRIT) _____ 

_____ b. Tactical communications support processor (TCSP) _____ 

_____ c. Multiple Subscriber Equipment (MSE) tactical packet network 

 (TPN) _____ 

_____ d. Direct wire line connection to AN/TYC-39 message switch _____ 

_____ e. Ultra-high frequency net radio protocol (UHF-NRP) _____ 

_____ f. Secure messaging and routing terminal (SMART) _____ 

_____ g. Multi function work stations _____ 

_____ h. Remote intelligence server _____ 

_____ i. Trusted workstation _____ 

_____ j. Web guards _____ 

_____ k. Web shields _____ 

_____ l. Trusted Guard System _____ 

_____ m. Tactical image processing subsystem (TIPS) _____ 

_____ n. Moving target indicator module _____ 
1 = Completely Disagree 3 = Generally Disagree 5 = Strongly Agree 9 = Unknown 

2 = Strongly Disagree 4 = Generally Agree 6 = Completely Agree 

206

Equipment Focus: Health Hazard

 Y/N Rating

 2. Have experienced or suspected one or more conditions 

 associated with operating the ___________ which I considered 

 unhealthy. _____ 

_____ a. TROJAN special purpose integrated remote intelligence 

 terminal (SPIRIT) _____ 

_____ b. Tactical communications support processor (TCSP) _____ 

_____ c. Multiple Subscriber Equipment (MSE) tactical packet network 

 (TPN) _____ 

_____ d. Direct wire line connection to AN/TYC-39 message switch _____ 

_____ e. Ultra-high frequency net radio protocol (UHF-NRP) _____ 

_____ f. Secure messaging and routing terminal (SMART) _____ 

_____ g. Multi function work stations _____ 

_____ h. Remote intelligence server _____ 

_____ i. Trusted workstation _____ 

_____ j. Web guards _____ 

_____ k. Web shields _____ 

_____ l. Trusted Guard System _____ 

 

_____ m. Tactical image processing subsystem (TIPS) _____ 

_____ n. Moving target indicator module _____ 
1 = Completely Disagree 3 = Generally Disagree 5 = Strongly Agree 9 = Unknown 

2 = Strongly Disagree 4 = Generally Agree 6 = Completely Agree 

207

Equipment Focus: System Safety

Y/N Rating

 3. Have noted safety problems (actual or potential) related to operation of 

 __________. 

_____ a. TROJAN special purpose integrated remote intelligence terminal 

 (SPIRIT) _____ 

_____ b. Tactical communications support processor (TCSP) _____ 

_____ c. Multiple Subscriber Equipment (MSE) tactical packet network 

 (TPN) _____ 

_____ d. Direct wire line connection to AN/TYC-39 message switch _____ 

_____ e. Ultra-high frequency net radio protocol (UHF-NRP) _____ 

_____ f. Secure messaging and routing terminal (SMART) _____ 

_____ g. Multi function work stations _____ 

_____ h. Remote intelligence server _____ 

_____ i. Trusted workstation _____ 

_____ j. Web guards _____ 

_____ k. Web shields _____ 

_____ l. Trusted Guard System _____ 

_____ m. Tactical image processing subsystem (TIPS) _____ 

_____ n. Moving target indicator module _____ 
1 = Completely Disagree 3 = Generally Disagree 5 = Strongly Agree 9 = Unknown 

2 = Strongly Disagree 4 = Generally Agree 6 = Completely Agree 

208

Equipment Focus: Training

Y/N Rating

 4. Have been required to perform one or more tasks or operations 

 with ________ for which your training was poor or inadequate. 

 a. TROJAN special purpose integrated remote intelligence _____ 

_____ terminal (SPIRIT) 

_____ b. Tactical communications support processor (TCSP) _____ 

_____ c. Multiple Subscriber Equipment (MSE) tactical packet network 

 (TPN) _____ 

_____ d. Direct wire line connection to AN/TYC-39 message switch _____ 

_____ e. Ultra-high frequency net radio protocol (UHF-NRP) _____ 

_____ f. Secure messaging and routing terminal (SMART) _____ 

_____ g. Multi function work stations _____ 

_____ h. Remote intelligence server _____ 

_____ i. Trusted workstation _____ 

_____ j. Web guards _____ 

_____ k. Web shields _____ 

_____ l. Trusted Guard System _____ 

_____ m. Tactical image processing subsystem (TIPS) _____ 

_____ n. Moving target indicator module _____ 
1 = Completely Disagree 3 = Generally Disagree 5 = Strongly Agree 9 = Unknown 

2 = Strongly Disagree 4 = Generally Agree 6 = Completely Agree 

209

Equipment Focus: Manpower

 

Y/N Rating

 5. Have been required to perform one or more tasks or operations with 

 ________that required an additional Team Member. 

_____ a. TROJAN special purpose integrated remote intelligence 

 terminal (SPIRIT) _____ 

_____ b. Tactical communications support processor (TCSP) _____ 

_____ c. Multiple Subscriber Equipment (MSE) tactical packet network 

 (TPN) _____ 

_____ d. Direct wire line connection to AN/TYC-39 message switch _____ 

_____ e. Ultra-high frequency net radio protocol (UHF-NRP) _____ 

_____ f. Secure messaging and routing terminal (SMART) _____ 

_____ g. Multi function work stations _____ 

_____ h. Remote intelligence server _____ 

_____ i. Trusted workstation _____ 

_____ j. Web guards _____ 

_____ k. Web shields _____ 

_____ l. Trusted Guard System _____ 

_____ m. Tactical image processing subsystem (TIPS) _____ 

_____ n. Moving target indicator module _____ 
1 = Completely Disagree 3 = Generally Disagree 5 = Strongly Agree 9 = Unknown 

2 = Strongly Disagree 4 = Generally Agree 6 = Completely Agree 

210

Equipment Focus: Personnel

Y/N Rating

 6. Have been required to perform one or more tasks or operations 

 with ______ that was outside of "normal" duties for my MOS and 

 skill level. 

 

_____ a. TROJAN special purpose integrated remote intelligence terminal 

 (SPIRIT) _____ 

_____ b. Tactical communications support processor (TCSP) _____ 

_____ c. Multiple Subscriber Equipment (MSE) tactical packet network 

 (TPN) _____ 

_____ d. Direct wire line connection to AN/TYC-39 message switch _____ 

_____ e. Ultra-high frequency net radio protocol (UHF-NRP) _____ 

_____ f. Secure messaging and routing terminal (SMART) _____ 

_____ g. Multi function work stations _____ 

_____ h. Remote intelligence server _____ 

_____ i. Trusted workstation _____ 

_____ j. Web guards _____ 

_____ k. Web shields _____ 

_____ l. Trusted Guard System _____ 

_____ m. Tactical image processing subsystem (TIPS) _____ 

_____ n. Moving target indicator module _____ 
1 = Completely Disagree 3 = Generally Disagree 5 = Strongly Agree 9 = Unknown 

2 = Strongly Disagree 4 = Generally Agree 6 = Completely Agree 

211

Equipment Focus: Human Factors Engineering--Procedures

Y/N Rating

 7. Have found procedures related to operation of ________ that 

 were unnecessarily difficult, complicated or "unfriendly. 

_____ a. TROJAN special purpose integrated remote intelligence terminal 

 (SPIRIT) _____ 

_____ b. Tactical communications support processor (TCSP) _____ 

_____ c. Multiple Subscriber Equipment (MSE) tactical packet network 

 (TPN) _____ 

_____ d. Direct wire line connection to AN/TYC-39 message switch _____ 

_____ e. Ultra-high frequency net radio protocol (UHF-NRP) _____ 

_____ f. Secure messaging and routing terminal (SMART) _____ 

_____ g. Multi function work stations _____ 

_____ h. Remote intelligence server _____ 

_____ i. Trusted workstation _____ 

_____ j. Web guards _____ 

_____ k. Web shields _____ 

_____ l. Trusted Guard System _____ 

_____ m. Tactical image processing subsystem (TIPS) _____ 

_____ n. Moving target indicator module _____ 
1 = Completely Disagree 3 = Generally Disagree 5 = Strongly Agree 9 = Unknown 

2 = Strongly Disagree 4 = Generally Agree 6 = Completely Agree 

212

Equipment Focus: Human Factors Engineering--Software

Y/N Rating

 8. Have found software problems associated with operation of 

 ________. 

_____ a. TROJAN special purpose integrated remote intelligence terminal 

 (SPIRIT) _____ 

_____ b. Tactical communications support processor (TCSP) _____ 

_____ c. Multiple Subscriber Equipment (MSE) tactical packet network 

 (TPN) _____ 

_____ d. Direct wire line connection to AN/TYC-39 message switch _____ 

_____ e. Ultra-high frequency net radio protocol (UHF-NRP) _____ 

_____ f. Secure messaging and routing terminal (SMART) _____ 

_____ g. Multi function work stations _____ 

_____ h. Remote intelligence server _____ 

_____ i. Trusted workstation _____ 

_____ j. Web guards _____ 

_____ k. Web shields _____ 

_____ l. Trusted Guard System _____ 

 

_____ m. Tactical image processing subsystem (TIPS) _____ 

_____ n. Moving target indicator module _____ 
1 = Completely Disagree 3 = Generally Disagree 5 = Strongly Agree 9 = Unknown 

2 = Strongly Disagree 4 = Generally Agree 6 = Completely Agree 

213

Equipment Focus: Human Factors Engineering--Placement

Y/N Rating

 9. Have found that placement of ________ (or their switches or 

 indicators) made access or use difficult. 

 

_____ a. TROJAN special purpose integrated remote intelligence terminal 

 (SPIRIT) _____ 

_____ b. Tactical communications support processor (TCSP) _____ 

_____ c. Multiple Subscriber Equipment (MSE) tactical packet network 

 (TPN) _____ 

_____ d. Direct wire line connection to AN/TYC-39 message switch _____ 

_____ e. Ultra-high frequency net radio protocol (UHF-NRP) _____ 

_____ f. Secure messaging and routing terminal (SMART) _____ 

_____ g. Multi function work stations _____ 

_____ h. Remote intelligence server _____ 

_____ i. Trusted workstation _____ 

_____ j. Web guards _____ 

_____ k. Web shields _____ 

_____ l. Trusted Guard System _____ 

_____ m. Tactical image processing subsystem (TIPS) _____ 

_____ n. Moving target indicator module _____ 
1 = Completely Disagree 3 = Generally Disagree 5 = Strongly Agree 9 = Unknown 

2 = Strongly Disagree 4 = Generally Agree 6 = Completely Agree 

214

Function Focus: Health Hazards

Y/N Rating

 10. Performing __________ jeopardized my safety or that of another 

 Team Member. 

_____ a1. communications front end with the CCS _____ 

_____ b1. an interface between Block II ACE All Source and 

 communication systems with the CCS _____ 

_____ c1. an interface between Block II ACE Single Source and 

 communication systems with the CCS _____ 

_____ d1. an interface between Block II ACE All Source and forward 

 tactical sensors with the CCS _____ 

 

_____ e1. an interface between Block II ACE Single Source and forward

 tactical sensors with the CCS _____ 

_____ f1. receipt and relay of information between (vertical) and within 

 (horizontal) echelons with the CCS _____ 

_____ g1. multi-level security (MLS)--SCI and collateral traffic with the 

 CCS _____ 

_____ h1. support to COMINT _____ 

_____ i1. support to ELINT _____ 

_____ j1. support to CI/HUMINT _____ 

_____ k1. support to Measurement and Signature Intelligence (MASINT) _____ 

_____ l1. support to Open Source Intelligence (OSINT) _____ 

_____ m1. support to All Source Fusion _____ 

_____ n1. support to Situation Development _____ 

_____ o1. support to IPB _____ 

_____ p1. support to Target Identification _____ 
1 = Completely Disagree 3 = Generally Disagree 5 = Strongly Agree 9 = Unknown 

2 = Strongly Disagree 4 = Generally Agree 6 = Completely Agree 

215

Function Focus: Health Hazards

Y/N Rating

 10 (cont.) Performing __________ jeopardized my safety or that of 

 another Team Member. 

_____ q1. support to Target Refinement _____ 

_____ r1. support to Target Nomination _____ 

_____ s1. support to ISR requirements _____ 

_____ t1. support to Asset Management Responsibilities _____ 
1 = Completely Disagree 3 = Generally Disagree 5 = Strongly Agree 9 = Unknown 

2 = Strongly Disagree 4 = Generally Agree 6 = Completely Agree 

216

Function Focus: Health Hazards

Y/N Rating

 11. Have experienced or suspected one or more conditions 

 associated with _________ which I considered unhealthy 

_____ a1. using the CCS for front end communications _____ 

_____ b1. using the CCS to interface between Block II ACE All Source 

 and communication systems with the CCS _____ 

_____ c1. using the CCS to interface between Block II ACE Single Source 

 and communication systems _____ 

_____ d1. using the CCS to interface between Block II ACE All Source 

 and forward tactical sensors _____ 

_____ e1. using the CCS to interface between Block II ACE Single Source 

 and forward tactical sensors _____ 

_____ f1. using the CCS to receive/relay of information between (vertical) 

 and within (horizontal) echelons _____ 

_____ g1. using the CCS to control multi-level security (MLS)--SCI and 

 collateral traffic _____ 

_____ h1. using the ACE to support COMINT _____ 

_____ i1. using the ACE to support ELINT _____ 

_____ j1. using the ACE to support CI/HUMINT _____ 

_____ k1. using the ACE to support Measurement and Signature 

 Intelligence (MASINT) _____ 

_____ l1. using the ACE to support Open Source Intelligence (OSINT) _____ 

_____ m1. using the ACE to support All Source Fusion _____ 

_____ n1. using the ACE to support Situation Development _____ 

_____ o1. using the ACE to support IPB _____ 
1 = Completely Disagree 3 = Generally Disagree 5 = Strongly Agree 9 = Unknown 

2 = Strongly Disagree 4 = Generally Agree 6 = Completely Agree 

217

Function Focus: Health Hazards

Y/N Rating

 11(cont) Have experienced or suspected one or more conditions 

 associated with _________ which I considered unhealthy 

 

_____ p1. using the ACE to support Target Identification _____ 

_____ q1. using the ACE to support Target Refinement _____ 

_____ r1. using the ACE to support Target Nomination _____ 

_____ s1. using the ACE to support ISR requirements _____ 

_____ t1. using the ACE to support Asset Management Responsibilities _____ 
1 = Completely Disagree 3 = Generally Disagree 5 = Strongly Agree 9 = Unknown 

2 = Strongly Disagree 4 = Generally Agree 6 = Completely Agree 

218

Function Focus: System Safety

Y/N Rating

 12. Have noted system safety problems (actual or potential) related to 

 operation of _________. 

_____ a1. the CCS for front end communications _____ 

_____ b1. the CCS to interface between Block II ACE All Source and 

 communication systems with the CCS _____ 

_____ c1. the CCS to interface between Block II ACE Single Source and 

 communication systems _____ 

_____ d1. the CCS to interface between Block II ACE All Source and 

 forward tactical sensors _____ 

_____ e1. the CCS to interface between Block II ACE Single Source and 

 forward tactical sensors _____ 

_____ f1. the CCS to receive/relay of information between (vertical) and 

 within (horizontal) echelons _____ 

_____ g1. the CCS to control multi-level security (MLS)--SCI and collateral 

 traffic _____ 

_____ h1. the ACE to support COMINT _____ 

_____ i1. the ACE to support ELINT _____ 

_____ j1. the ACE to support CI/HUMINT _____ 

_____ k1. the ACE to support Measurement and Signature Intelligence 

 (MASINT) _____ 

_____ l1. the ACE to support Open Source Intelligence (OSINT) _____ 

_____ m1. the ACE to support All Source Fusion _____ 

_____ n1. the ACE to support Situation Development _____ 

_____ o1. the ACE to support IPB _____ 
1 = Completely Disagree 3 = Generally Disagree 5 = Strongly Agree 9 = Unknown 

2 = Strongly Disagree 4 = Generally Agree 6 = Completely Agree 

219

Function Focus: System Safety

Y/N Rating

 12(cont) Have noted system safety problems (actual or potential) related 

 to operation of _________. 

_____ p1. the ACE to support Target Identification _____ 

_____ q1. the ACE to support Target Refinement _____ 

_____ r1. the ACE to support Target Nomination _____ 

_____ s1. the ACE to support ISR requirements _____ 

_____ t1. the ACE to support Asset Management Responsibilities _____ 
1 = Completely Disagree 3 = Generally Disagree 5 = Strongly Agree 9 = Unknown 

2 = Strongly Disagree 4 = Generally Agree 6 = Completely Agree 

220

Function Focus: Training

Y/N Rating

 13. Have been required to perform one or more tasks or operations 

 with ______________ for which my training was poor or inadequate. 

_____ a1. the CCS for front end communications _____ 

_____ b1. the CCS to interface between Block II ACE All Source and 

 communication systems with the CCS _____ 

_____ c1. the CCS to interface between Block II ACE Single Source and 

 communication systems _____ 

_____ d1. the CCS to interface between Block II ACE All Source and 

 forward tactical sensors _____ 

_____ e1. the CCS to interface between Block II ACE Single Source and 

 forward tactical sensors _____ 

_____ f1. the CCS to receive/relay of information between (vertical) and 

 within (horizontal) echelons _____ 

_____ g1. the CCS to control multi-level security (MLS)--SCI and collateral 

 traffic _____ 

_____ h1. the ACE to support COMINT _____ 

_____ i1. the ACE to support ELINT _____ 

_____ j1. the ACE to support CI/HUMINT _____ 

_____ k1. the ACE to support Measurement and Signature Intelligence 

 (MASINT) _____ 

_____ l1. the ACE to support Open Source Intelligence (OSINT) _____ 

_____ m1. the ACE to support All Source Fusion _____ 

_____ n1. the ACE to support Situation Development _____ 

_____ o1. the ACE to support IPB _____ 
1 = Completely Disagree 3 = Generally Disagree 5 = Strongly Agree 9 = Unknown 

2 = Strongly Disagree 4 = Generally Agree 6 = Completely Agree 

221

Function Focus: Training

Y/N Rating

 13(cont). Have been required to perform one or more tasks or 

 operations with ______________ for which my training was poor 

 or inadequate 

_____ p1. the ACE to support Target Identification _____ 

_____ q1. the ACE to support Target Refinement _____ 

_____ r1. the ACE to support Target Nomination _____ 

_____ s1. the ACE to support ISR requirements _____ 

_____ t1. the ACE to support Asset Management Responsibilities _____ 
1 = Completely Disagree 3 = Generally Disagree 5 = Strongly Agree 9 = Unknown 

2 = Strongly Disagree 4 = Generally Agree 6 = Completely Agree 

222

Function Focus: Manpower

Y/N Rating

 14. Have been required to perform one or more tasks or operations 

 with ___________ that required an additional Team Member. 

_____ a1. the CCS for front end communications _____ 

_____ b1. the CCS to interface between Block II ACE All Source and 

 communication systems with the CCS _____ 

_____ c1. the CCS to interface between Block II ACE Single Source and 

 communication systems _____ 

_____ d1. the CCS to interface between Block II ACE All Source and 

 forward tactical sensors _____ 

_____ e1. the CCS to interface between Block II ACE Single Source and 

 forward tactical sensors _____ 

_____ f1. the CCS to receive/relay of information between (vertical) and 

 within (horizontal) echelons _____ 

_____ g1. the CCS to control multi-level security (MLS)--SCI and collateral 

 traffic _____ 

_____ h1. the ACE to support COMINT _____ 

_____ i1. the ACE to support ELINT _____ 

_____ j1. the ACE to support CI/HUMINT _____ 

_____ k1. the ACE to support Measurement and Signature Intelligence 

 (MASINT) _____ 

_____ l1. the ACE to support Open Source Intelligence (OSINT) _____ 

_____ m1. the ACE to support All Source Fusion _____ 

_____ n1. the ACE to support Situation Development _____ 

_____ o1. the ACE to support IPB _____ 
1 = Completely Disagree 3 = Generally Disagree 5 = Strongly Agree 9 = Unknown 

2 = Strongly Disagree 4 = Generally Agree 6 = Completely Agree 

223

Function Focus: Manpower

Y/N Rating

 

 14 (cont). Have been required to perform one or more tasks or 

 operations with ___________ that required an additional Team 

 Member. 

_____ p1. the ACE to support Target Identification _____ 

_____ q1. the ACE to support Target Refinement _____ 

_____ r1. the ACE to support Target Nomination _____ 

_____ s1. the ACE to support ISR requirements _____ 

_____ t1. the ACE to support Asset Management Responsibilities _____ 
1 = Completely Disagree 3 = Generally Disagree 5 = Strongly Agree 9 = Unknown 

2 = Strongly Disagree 4 = Generally Agree 6 = Completely Agree 

224

Function Focus: Personnel

Y/N Rating

 15. Required to perform tasks or operations with _________ 

 that was outside of "normal" duties for my MOS and skill level. 

_____ a1. the CCS for front end communications _____ 

_____ b1. the CCS to interface between Block II ACE All Source and 

 communication systems with the CCS _____ 

_____ c1. the CCS to interface between Block II ACE Single Source and 

 communication systems _____ 

_____ d1. the CCS to interface between Block II ACE All Source and 

 forward tactical sensors _____ 

_____ e1. the CCS to interface between Block II ACE Single Source 

 and forward tactical sensors _____ 

_____ f1. the CCS to receive/relay of information between (vertical) and 

 within (horizontal) echelons _____ 

_____ g1. the CCS to control multi-level security (MLS)--SCI and collateral 

 traffic _____ 

_____ h1. the ACE to support COMINT _____ 

_____ i1. the ACE to support ELINT _____ 

_____ j1. the ACE to support CI/HUMINT _____ 

_____ k1. the ACE to support Measurement and Signature Intelligence 

 (MASINT) _____ 

_____ l1. the ACE to support Open Source Intelligence (OSINT) _____ 

_____ m1. the ACE to support All Source Fusion _____ 

_____ n1. the ACE to support Situation Development _____ 

_____ o1. the ACE to support IPB _____ 
1 = Completely Disagree 3 = Generally Disagree 5 = Strongly Agree 9 = Unknown 

2 = Strongly Disagree 4 = Generally Agree 6 = Completely Agree 

225

Function Focus: Personnel

Y/N Rating

 15 (cont) Required to perform tasks or operations with _________ 

 that was outside of "normal" duties for my MOS and skill level. 

_____ p1. the ACE to support Target Identification _____ 

_____ q1. the ACE to support Target Refinement _____ 

_____ r1. the ACE to support Target Nomination _____ 

_____ s1. the ACE to support ISR requirements _____ 

_____ t1. the ACE to support Asset Management Responsibilities _____ 
1 = Completely Disagree 3 = Generally Disagree 5 = Strongly Agree 9 = Unknown 

2 = Strongly Disagree 4 = Generally Agree 6 = Completely Agree 

226

Function Focus: Human Factors Engineering--Procedures

 Y/N Rating

 16. Have found procedures related to operation of ____________ 

 that were unnecessarily difficult, complicated or "unfriendly". 

_____ a1. the CCS for front end communications _____ 

_____ b1. the CCS to interface between Block II ACE All Source and 

 communication systems with the CCS _____ 

_____ c1. the CCS to interface between Block II ACE Single Source and 

 communication systems _____ 

_____ d1. the CCS to interface between Block II ACE All Source and 

 forward tactical sensors _____ 

_____ e1. the CCS to interface between Block II ACE Single Source and 

 forward tactical sensors _____ 

_____ f1. the CCS to receive/relay of information between (vertical) and 

 within (horizontal) echelons _____ 

_____ g1. the CCS to control multi-level security (MLS)--SCI and collateral 

 traffic _____ 

_____ h1. the ACE to support COMINT _____ 

_____ i1. the ACE to support ELINT _____ 

_____ j1. the ACE to support CI/HUMINT _____ 

_____ k1. the ACE to support Measurement and Signature Intelligence 

 (MASINT) _____ 

_____ l1. the ACE to support Open Source Intelligence (OSINT) _____ 

_____ m1. the ACE to support All Source Fusion _____ 

_____ n1. the ACE to support Situation Development _____ 

_____ o1. the ACE to support IPB _____ 
1 = Completely Disagree 3 = Generally Disagree 5 = Strongly Agree 9 = Unknown 

2 = Strongly Disagree 4 = Generally Agree 6 = Completely Agree 

227

Function Focus: Human Factors Engineering--Procedures

Y/N Rating

 

 16 (cont) Have found procedures related to operation of ____________ 

 that were unnecessarily difficult, complicated or "unfriendly". 

_____ p1. the ACE to support Target Identification _____ 

_____ q1. the ACE to support Target Refinement _____ 

_____ r1. the ACE to support Target Nomination _____ 

_____ s1. the ACE to support ISR requirements _____ 

_____ t1. the ACE to support Asset Management Responsibilities _____ 
1 = Completely Disagree 3 = Generally Disagree 5 = Strongly Agree 9 = Unknown 

2 = Strongly Disagree 4 = Generally Agree 6 = Completely Agree 

228

Function Focus: Human Factors Engineering: Software

 Y/N Rating

 17. Have found software problems associated with operation of 

 ________. 

_____ a1. the CCS for front end communications _____ 

_____ b1. the CCS to interface between Block II ACE All Source and 

 communication systems with the CCS _____ 

_____ c1. the CCS to interface between Block II ACE Single Source and 

 communication systems _____ 

_____ d1. the CCS to interface between Block II ACE All Source and 

 forward tactical sensors _____ 

_____ e1. the CCS to interface between Block II ACE Single Source and 

 forward tactical sensors _____ 

_____ f1. the CCS to receive/relay of information between (vertical) and 

 within (horizontal) echelons _____ 

_____ g1. the CCS to control multi-level security (MLS)--SCI and collateral 

 traffic _____ 

_____ h1. the ACE to support COMINT _____ 

_____ i1. the ACE to support ELINT _____ 

_____ j1. the ACE to support CI/HUMINT _____ 

_____ k1. the ACE to support Measurement and Signature Intelligence 

 (MASINT) _____ 

_____ l1. the ACE to support Open Source Intelligence (OSINT) _____ 

_____ m1. the ACE to support All Source Fusion _____ 

_____ n1. the ACE to support Situation Development _____ 

_____ o1. the ACE to support IPB _____ 
1 = Completely Disagree 3 = Generally Disagree 5 = Strongly Agree 9 = Unknown 

2 = Strongly Disagree 4 = Generally Agree 6 = Completely Agree 

229

Function Focus: Human Factors Engineering: Software

Y/N Rating

 

 17. Have found software problems associated with operation of 

 ________. 

_____ p1. the ACE to support Target Identification _____ 

 

_____ q1. the ACE to support Target Refinement _____ 

_____ r1. the ACE to support Target Nomination _____ 

_____ s1. the ACE to support ISR requirements _____ 

_____ t1. the ACE to support Asset Management Responsibilities _____ 
1 = Completely Disagree 3 = Generally Disagree 5 = Strongly Agree 9 = Unknown 

2 = Strongly Disagree 4 = Generally Agree 6 = Completely Agree 

230

Function Focus: Human Factors Engineering: Placement

Y/N Rating

 18. Have found that placement of equipment, switches or indicators 

 in ________________ made access or use difficult. 

_____ a1. the CCS for front end communications _____ 

_____ b1. the CCS to interface between Block II ACE All Source and 

 communication systems with the CCS _____ 

_____ c1. the CCS to interface between Block II ACE Single Source and 

 communication systems _____ 

_____ d1. the CCS to interface between Block II ACE All Source and 

 forward tactical sensors _____ 

_____ e1. the CCS to interface between Block II ACE Single Source and 

 forward tactical sensors _____ 

_____ f1. the CCS to receive/relay of information between (vertical) and 

 within (horizontal) echelons _____ 

_____ g1. the CCS to control multi-level security (MLS)--SCI and collateral 

 traffic _____ 

_____ h1. the ACE to support COMINT _____ 

_____ i1. the ACE to support ELINT _____ 

_____ j1. the ACE to support CI/HUMINT _____ 

_____ k1. the ACE to support Measurement and Signature Intelligence 

 (MASINT) _____ 

_____ l1. the ACE to support Open Source Intelligence (OSINT) _____ 

_____ m1. the ACE to support All Source Fusion _____ 

_____ n1. the ACE to support Situation Development _____ 

_____ o1. the ACE to support IPB _____ 

_____ p1. the ACE to support Target Identification _____ 
1 = Completely Disagree 3 = Generally Disagree 5 = Strongly Agree 9 = Unknown 

2 = Strongly Disagree 4 = Generally Agree 6 = Completely Agree 

231

Function Focus: Human Factors Engineering: Placement

 Y/N Rating

 18. Have found that placement of equipment, switches or indicators 

 in ________________ made access or use difficult. 

_____ q1. the ACE to support Target Refinement _____ 

_____ r1. the ACE to support Target Nomination _____ 

_____ s1. the ACE to support ISR requirements _____ 

_____ t1. the ACE to support Asset Management Responsibilities _____ 
1 = Completely Disagree 3 = Generally Disagree 5 = Strongly Agree 9 = Unknown 

2 = Strongly Disagree 4 = Generally Agree 6 = Completely Agree 

232

_____________________________________________________________________________ 

_____________________________________________________________________________ 

_____________________________________________________________________________ 

_____________________________________________________________________________ 

_____________________________________________________________________________ 

_____________________________________________________________________________ 

_____________________________________________________________________________ 

_____________________________________________________________________________ 

_____________________________________________________________________________ 

_____________________________________________________________________________ 

_____________________________________________________________________________ 

_____________________________________________________________________________ 

_____________________________________________________________________________ 

_____________________________________________________________________________ 

_____________________________________________________________________________ 

_____________________________________________________________________________ 

_____________________________________________________________________________ 

_____________________________________________________________________________ 

_____________________________________________________________________________ 

_____________________________________________________________________________ 

_____________________________________________________________________________ 
233

Appendix I. Noise and Temperature Measurements in and Around 

the HETS M1070 Tractor

 

This appendix appears in its original form, without editorial change. 
234

TABLE D-15. ENVIRONMENTAL DATA BASED ON 

AMBIENT TEMPERATURES

Reading 

Sample 

Size Mean Std dev 

Ambient temperature of 51 to 60

Cab temperature ( F) 225 68.36 7.82 

Cab noise (dB) 225 78.27 5.49 

Ambient temperature 225 52.76 3.49 

Relative humidity (pct) 225 61.00 1.72 

 

Ambient temperature of 61 to 70F 

Cab temperature ( F) 180 79.91 6.89 

Cab noise (dB) 180 79.63 5.26 

Ambient temperature 180 68.08 2.26 

Relative humidity (pct) 180 57.08 1.39 

 

Ambient temperature of 71to 80 F 

Cab temperature ( F) 180 89.97 5.69 

Cab noise (dB) 180 79.49 4.70 

Ambient temperature 180 76.33 2.63 

Relative humidity (pct) 180 56.00 0.00 

 

Ambient temperature of 81 to 90 F 

Cab temperature ( F) 180 94.52 6.29 

Cab noise (dB) 180 78.88 5.43 

Ambient temperature 180 82.33 1.38 

Relative humidity (pct) 180 55.67 1.25 

NOTE: Army standards state that hearing protection is recommended 

When noise levels exceed 85 decibels. Temperature is considered 

excessive when internal temperatures are 10 higher than the 

ambient temperature. 

TABLE D-16. M1070 TRACTOR (CAB) TEMPERATURE AND 

NOISE FOR EACH WINDOW POSITION 

 

Temperature ( F) Noise levels (db)

Window 

position 

Sample 

size Mean Std dev 

Sample 

size Mean Std dev 

 

Closed 255 85.67 11.58 255 78.55 5.05 

Half open 255 83.30 12.17 255 78.93 5.13 

Open 255 77.99 12.18 255 79.56 5.57 

 
235

TABLE D-17. M1070 TRACTOR (CAB) TEMPERATURE AND 

NOISE WHILE TRAVELING AT DIFFERENT SPEEDS 

Reading 

Sample 

Size Mean Std dev 

0 miles per hour speed 

Cab temperature ( F) 255 79.11 15.23 

Noise level (dB) 255 71.51 2.00 

20 miles per hour speed 

Cab temperature ( F) 180 83.04 10.91 

Noise level (dB) 180 81.99 1.96 

30 miles per hour speed 

Cab temperature ( F) 180 83.56 10.72 

Noise level (dB) 180 82.07 2.10 

40 miles per hour speed 

Cab temperature ( F) 180 84.38 10.56 

Noise level (dB) 180 82.36 2.16 

TABLE D-18. M1070 TRACTOR TEMPERATURE AND NOISE 

LEVEL FOR DIFFERENT POSITIONS IN CAB 

Temperature ( F) Noise levels (dB) 

Position 

Sample 

Size Mean Std dev 

Sample 

Size Mean Std dev 

Floor (near driver's 

 feet) 153 82.95 12.20 153 79.35 5.03 

Driver's waist level 153 82.19 12.11 153 78.59 5.29 

Driver's head level 

 (window side) 153 80.69 12.06 153 78.84 5.56 

Driver's head level 

 (doghouse side) 153 84.83 13.28 153 79.28 4.88 

Floor near assistant 

 driver's feet 153 80.95 11.95 153 79.00 5.55 
236
237

Appendix J. MANPRINT Biographical Questionnaire: Enhanced Position 

Location Reporting System (EPLRS)

 This appendix appears in its original form, without editorial change. 
238

ENHANCED POSITION LOCATION REPORTING SYSTEM (EPLRS) 

PLAYER/MAINTAINER 

MANPRINT BIOGRAPHICAL QUESTIONNAIRE 

INFORMATION PRIVACY ACT STATEMENT 

A. Authority: 5 USC 301, 10 USC 3012, Authority for Secretary of the Army to 

Issue ARs; 44 USC 3101, Authority for Collecting Necessary Data. 

B. Principal Purpose: To collect data to evaluate the effectiveness of 

soldiers with different backgrounds to operate and maintain EPLRS. 

C. Routine Uses: The data collected are to be used for research purposes 

only. They will not become a part of any individual's record and will not be 

used in whole or part in making any determination about an individual. Full 

confidentiality of responses will be maintained in the processing of these 

data. 

D. Mandatory or Voluntary Disclosure and Effect on Individual Not Providing 

Information: Voluntary - Your participation in this research is strictly 

voluntary. Individuals are encouraged to provide complete and accurate 

information in the interests of the research, but there will be no effect on 

individuals not providing all or any part of the information. 

 This notice may be detached from the rest of this form and retained by 

 the individual answering the questionnaire if so desired. 
239

ENHANCED POSITION LOCATION REPORTING SYSTEM (EPLRS) 

MANPRINT BIOGRAPHICAL PROFILE 

Purpose: This form is important because it is about you! The data you provide will help Department of Defense 

decision-makers evaluate how effectively EPLRS has been designed for soldiers like you. To do this, we need you 

to complete this for accurately. None of your individual data will be provided to your chain of command nor

entered in your personnel files. Your responses will be kept confidential. 

Instructions: Read each item carefully and enter your response in the item space provided or circle the appropriate 

response. 

1. NAME: ________________________________ 

 (Please print) 

2. PIN NO. ___ ___ ___ 

3. MOS: 

 a. Primary ___ ___ ___ 

 b. Secondary ___ ___ ___ 

 c. Duty ___ ___ ___ 

4. Time in primary MOS: _____________ 

 (months) 

5. Duty in test: 

 1 = Net Control Station (NCS) Operator 

 2= Net Control System(NCS) Maintainer 

 3= Radio Set (RS) Operator 

 4= Radio Set (RS) Maintainer 

 5= EPLRS Ground Reference Unit (EGRU) 

 Operator 

 6= EPLRS Test Set (ETS) DS Maintenance 

 Support 

 7= System Management 

6. Paygrade: Please indicate: 

 E- ___ 

 OR 

 O- ___ 

7. a. Time in Grade ____________ 

 (months) 

 b. Promotable Status: 

 1 = Yes 2 = No 

8. Reenlistment 

 a. Plan to reenlist: 

 1 = Yes 2 = No 

 b. Are you sure? 

 1 = Yes 2 = No 

9. Gender: 1 = Male 2 = Female 10. In my immediate family, I was the 

 1 = Only child 

 2 = Oldest child 

 3 = Second or third oldest 

 4 = Fourth or fifth oldest 

 5 = More than fifth oldest 

11. Marital Status: 

 1 = Single 

 2 = Married 

 3 = Separated 

 4 = Divorced 

12. Race: 

 1 = White 

 2 = Black 

 3 = Hispanic 

 4 = American Indian/Alaskan 

 5 = Asian/Pacific Islander 

 6 = Other/unknown 
240

13. Education: 

 1 = No high school degree 

 2 = High school diploma 

 3 = GED 

 4 = Some college (1-2 yrs) /technical school 

 5 = Two year college degree 

 6 = Three or more years college, no degree 

 7 = Four year college degree 

14. Place and Date of Birth: 

 a. City ________________________ 

 

 b. State _______________________ 

 c. Country _____________________ 

 d. Date ____ ____ ____ 

 dd mm yy 

15. Size of community I grew up in: 

 1 = Rural 

 2 = Town (less than 10,000) 

 3 = Town (10,000 - 90,000) 

 4 = City (90,000 - 200,000) 

 5 = City (200,001 - 1 million) 

 6 = More than 1 million 

16. Handedness - write with: 

 1 = Right 

 2 = Left 

 3 = Both equally well 

17. Eye Dominance: 

 1 = Right 2 = Left 3 = Both 

18. Height (inches) __________________ 

19. Weight (pounds): __________________ 20. Wear glasses: 

 1 = Yes 2 = No 3 = Reading only 

21. Hearing: 

 a. Any impairment? 1 = Yes 2 = No 

 b. Corrected? 1 = Yes 2 = No 

 c. If yes to b, how?______________________ 

 ___________________________________ 

22. Physical Profile: 

 a. Do you have one? 

 1 = Yes 2 = No 

 b. If Yes to a, what is it for? 

 ___________________________________ 

 c. If Yes to a, is it: 

 1 = Temporary 2 = Permanent 

23. Father's occupation while growing up: 

_______________________________________ 

24. Mother's occupation while growing up: 

________________________________________ 

25. a. Language used in home while growing up? 

 1 = Only English 

 2 = Mainly English 

 3 = Occasionally English 

 4 = Seldom English 

 5 = Never English 

 b. Other language used? 

 __________________________________ 
241

26. Compared to others in high school, I thought of myself as: 

 1 = Smarter than most other students 

 2 = As smart as most students 

 3 = Not as smart as most students 

27. Before joining the military, how often did you read books or magazines of your own choosing? (Do not 

 Include newspapers or comics.) 

 1 = Practically never 

 2 = Seldom 

 3 = Occasionally, such as once per week 

 4 = Often, such as at least once a day 

 5 = Very often, such as more than once a day 

28. In general, I'm the type of person who prefers to do a job: 

 1 = Usually by myself 

 2 = Often by myself 

 3 = Sometimes by myself, sometimes with others 

 4 = Often with others 

 5 = Usually with others 

29. Who succeeds often depends on luck, such as, who was in the right place at the right time: 

 1 = That's almost always true 

 2 = That's often true 

 3 = Sometimes that's true 

 4 = That's seldom true 

 5 = That's never true 

30. Compared to others in athletic ability, I am: 

 1 = Less athletic than most people in my age group 

 2 = Average in athletic ability 

 3 = Slightly more athletic than most people in my age group 

 4 = In the top 15% of people in my age group 

 5 = In the top 1% of people in my age group 

31. Compared to other, I generally do my job: 

 1 = Slower than most performers 

 2 = Slightly slower than most performers 

 3 = About average 

 4 = Slightly faster than most performers 

 5 = Faster than most performers 

32. Compared to others, I generally do my job: 

 1 = More correctly than most performers 

 2 = Slightly more correctly than most performers 

 3 =About average 

 4 = Slightly less correctly than most performers 

 5 = Less correctly than most performers 
242

33. To feel really good mentally, I require: 

 1 = 4 hours sleep or less 

 2 = 5 to 7 hours sleep 

 3 = 8 hours sleep 

 4 = 9 hours sleep 

 5 = 10 or more hours sleep 

34. To be successful at difficult jobs means sometimes you have to take risks. 

 1 = Strongly disagree 

 2 = Disagree 

 3 = I am uncertain 

 4 = Agree 

 5 = Strongly agree 

35. Nothing makes me more angry than equipment which does NOT work reliably. 

 1 = Strongly disagree 

 2 = Disagree 

 3 = I am uncertain 

 4 = Agree 

 5 = Strongly agree 

36. It is hard to exert job effort if you work alone. 

 1 = Strongly disagree 

 2 = Disagree 

 3 = I am uncertain 

 4 = Agree 

 5 = Strongly agree 

37. In important matters, I usually: 

 1 = Rely on my friends for information 

 2 = Rely on my family for information 

 3 = Rely on some knowledgeable expert 

 4 = Rely on myself to find things out 

38. How well I do my new job significantly impacts the jobs of others. 

 1 = Strongly disagree 

 2 = Disagree 

 3 = I am uncertain 

 4 = Agree 

 5 = Strongly agree 

39. Most people I work with would rather take more time to do a job right, than less time to do a "quick and 

dirty, it's good enough job" 

 1 = Strongly disagree 

 2 = Disagree 

 3 = I am uncertain 

 4 = Agree 

 5 = Strongly agree 

. 
243

40. Most jobs are dull and boring. 

 1 = Strongly disagree 

 2 = Disagree 

 3 = I am uncertain 

 4 = Agree 

 5 = Strongly agree 

41. Only I am responsible for what I will be. 

 1 = Strongly disagree 

 2 = Disagree 

 3 = I am uncertain 

 4 = Agree 

 5 = Strongly agree 

42. I know if I persist, I can make unreliable equipment work properly 

 1 = Strongly disagree 

 2 = Disagree 

 3 = I am uncertain 

 4 = Agree 

 5 = Strongly agree 

43. I'd rather be recognized as a "star" on a losing team than an unrecognized member of a winning 

 team. 

 1 = Strongly disagree 

 2 = Disagree 

 3 = I am uncertain 

 4 = Agree 

 5 = Strongly agree 

44. People make their own breaks in life. 

 1 = Strongly disagree 

 2 = Disagree 

 3 = I am uncertain 

 4 = Agree 

 5 = Strongly agree 

45. My greatest source of job satisfaction comes primarily from: 

 1 = The people I work with 

 2 = The high tech equipment 

 3 = The sense of accomplishing something important 

 4 = Being successful 

 5 = The paycheck 

46. When traveling to a new city, it's easy to get lost even when you have a map. 

 1 = Strongly disagree 

 2 = Disagree 

 3 = I am uncertain 

 4 = Agree 

 5 = Strongly agree 
244

47. People count on me; I feel like I'm really needed at work. 

 1 = Strongly disagree 

 2 = Disagree 

 3 = I am uncertain 

 4 = Agree 

 5 = Strongly agree 

48. My greatest job strength is my: 

 1 = Physical endurance 

 2 = Capability to learn 

 3 = "Can-do" attitude 

 4 = Level of knowledge 

 5 = Level of "know-how" experience 

49. If money or family obligations were not an obstacle, I would really like to be a: 

 1 = Race car driver 

 2 = Helicopter or jet pilot 

 3 = An accountant 

 4 = Computer scientist 

 5 = Lawyer 

 6 = Physician 

 7 = Scientist or engineer 

 8 = Professor 

 9 = Other_______________________________________________________ 

50. I intend to achieve the following end of training course score: 

 1 = 100% 

 2 = 90-99% 

 3 = 85-89% 

 4 = 80-84% 

 5 = 70-79% 

 6 = below 70% 

51. How happy are you in being selected to participate in this test? 

 1 = Very happy 

 2 = Happy 

 3 = Neither happy or unhappy 

 4 = Unhappy 

 5 = Very unhappy 

THANK YOU FOR YOUR COOPERATION 
NO. OF 

COPIES ORGANIZATION 

245

 1 DEFENSE TECHNICAL 

 (PDF INFORMATION CTR 

 only) DTIC OCA 

 8725 JOHN J KINGMAN RD 

 STE 0944 

 FORT BELVOIR VA 22060-6218 

 1 DIRECTOR 

 US ARMY RESEARCH LAB 

 IMNE ALC IMS 

 2800 POWDER MILL RD 

 ADELPHI MD 20783-1197 

 1 DIRECTOR 

 US ARMY RESEARCH LAB 

 AMSRD ARL CI OK TL 

 2800 POWDER MILL RD 

 ADELPHI MD 20783-1197 

 1 DIRECTOR 

 US ARMY RESEARCH LAB 

 AMSRD ARL CI OK PE 

 2800 POWDER MILL RD 

 ADELPHI MD 20783-1197 

ABERDEEN PROVING GROUND

 1 DIR USARL 

 AMSRD ARL CI OK TP (BLDG 4600) 
NO. OF NO. OF 

COPIES ORGANIZATION COPIES ORGANIZATION

246

 1 ARMY RSCH LABORATORY - HRED 

 AMSRD ARL HR ML J MARTIN 

 MYER CENTER RM 2D311 

 FT MONMOUTH NJ 07703-5601 

 1 ARMY RSCH LABORATORY - HRED 

 AMSRD ARL HR MZ A DAVISON 

 320 MANSCEN LOOP STE 115 

 FT LEONARD WOOD MO 65473 

 1 ARMY RSCH LABORATORY - HRED 

 AMSRD ARL HR MD T COOK 

 BLDG 5400 RM C242 

 REDSTONE ARSENAL AL 35898-7290 

 1 COMMANDANT USAADASCH 

 ATSA CD 

 AMSRD ARL HR ME DR HAWLEY 

 5800 CARTER RD 

 FT BLISS TX 79916-3802 

 1 ARMY RSCH LABORATORY - HRED 

 AMSRD ARL HR MM DR V J RICE 

 BLDG 4011 RM 217 

 1750 GREELEY RD 

 FT SAM HOUSTON TX 78234-5002 

 1 ARMY RSCH LABORATORY - HRED 

 AMSRD ARL HR MG R SPINE 

 BLDG 333 

 PICATINNY ARSENAL NJ 07806-5000 

 1 ARL HRED ARMC FLD ELMT 

 AMSRD ARL HR MH C BURNS 

 BLDG 1467B RM 336 

 THIRD AVE 

 FT KNOX KY 40121 

 1 ARMY RSCH LABORATORY - HRED 

 AWC FIELD ELEMENT 

 AMSRD ARL HR MJ D DURBIN 

 BLDG 4506 (DCD) RM 107 

 FT RUCKER AL 36362-5000 

 1 ARMY RSCH LABORATORY - HRED 

 AMSRD ARL HR MK MR J REINHART 

 10125 KINGMAN RD 

 FT BELVOIR VA 22060-5828 

 1 ARMY RSCH LABORATORY - HRED 

 AMSRD ARL HR MV HQ USAOTC 

 S MIDDLEBROOKS 

 91012 STATION AVE RM 348 

 FT HOOD TX 76544-5073 

 1 ARMY RSCH LABORATORY - HRED 

 AMSRD ARL HR MY M BARNES 

 2520 HEALY AVE 

 STE 1172 BLDG 51005 

 FT HUACHUCA AZ 85613-7069 

 1 ARMY RSCH LABORATORY - HRED 

 AMSRD ARL HR MP D UNGVARSKY 

 POPE HALL BLDG 4709 

 BCBL 806 HARRISON DR 

 FT LEAVENWORTH KS 66027-2302 

 1 ARMY RSCH LABORATORY - HRED 

 AMSRD ARL HR MJF J HANSBERGER 

 JFCOM JOINT EXPERIMENTATION J9 

 JOINT FUTURES LAB 

 115 LAKEVIEW PKWY STE B 

 SUFFOLK VA 23435 

 1 ARMY RSCH LABORATORY - HRED 

 AMSRD ARL HR MQ M R FLETCHER 

 US ARMY SBCCOM 

 NATICK SOLDIER CTR 

 AMSRD NSC WS E BLDG 3 RM 343 

 NATICK MA 01760-5020 

 1 ARMY RSCH LABORATORY-HRED 

 AMSRD ARL HR MT J CHEN 

 12423 RESEARCH PKWY 

 ORLANDO FL 32826 

 1 ARMY RSCH LABORATORY - HRED 

 AMSRD ARL HR MT C KORTENHAUS 

 12350 RESEARCH PKWY 

 ORLANDO FL 32826 

 1 ARMY RSCH LABORATORY - HRED 

 AMSRD ARL HR MS C MANASCO 

 SIGNAL TOWERS 

 BLDG 29808A RM 303 

 FT GORDON GA 30905-5233 

 1 ARMY RSCH LABORATORY - HRED 

 AMSRD ARL HR MU M SINGAPORE 

 6501 E 11 MILE RD MS 284 

 BLDG 200A 2ND FL RM 2104 

 WARREN MI 48397-5000 

 1 ARMY RSCH LABORATORY - HRED 

 AMSRD ARL HR MF C HERNANDEZ 

 2421 NW AUSTIN RD STE 220 

 FT SILL OK 73503-9042 
NO. OF 

COPIES ORGANIZATION 

247

 1 ARMY RSCH LABORATORY - HRED 

 AMSRD ARL HR MW E REDDEN 

 BLDG 4 ROOM 332 

 FT BENNING GA 31905-5400 

 1 ARMY RSCH LABORATORY - HRED 

 AMSRD ARL HR MN R SPENCER 

 DCSFDI HF 

 HQ USASOC BLDG E2929 

 FT BRAGG NC 28310-5000 

 1 ARMY G1 

 (CD DAPE MR B KNAPP 

 only) 300 ARMY PENTAGON ROOM 2C489 

 WASHINGTON DC 20310-0300 

ABERDEEN PROVING GROUND

 2 DIR USARL 

 AMSRD ARL CI OK TP 

 S FOPPIANO 

 AMSRD ARL HR MR 

 F PARAGALLO 
NO. OF NO. OF 

COPIES ORGANIZATION COPIES ORGANIZATION

248

 5 ARMY RSCH LABORATORY - HRED 

 AMSRD ARL HR MV HQ USAOTC 

 O HEUCKEROTH 

 91012 STATION AVE RM 348 

 FT HOOD TX 76544-5073 

 1 ARMY RSCH LABORATORY - HRED 

 AMSRD ARL HR MM 

 R BATEMAN 

 AMEDD FIELD OFFICE 

 1750 GREELY RD 

 BLDG 4011 STE 213 

 FT SAM HOUSTIN TX 75234-5002 

 3 DEFNS MANPOWER DATA CTR 

 DOD CTR 

 BRANCH CHIEF 

 FOR PERSONNEL & MANPOWER 

 M RUDOLPH 

 MONTEREY BAY 

 SEASIDE CA 93955-6771 

 1 ARMY RSCH LABORATORY - HRED 

 AMSRD ARL HR MY 

 B HUNN 

 2520 HEALY AVE 

 BLDG 51005 STE 1172 

 FIELD ELEMENT HUAC 

 FT HUACHUCA AZ 85613-7069 

 1 ARMY RSCH LABORATORY – HRED 

 AMSRD ARL HR ML 

 P SAVAGE-KNEPSHIELD 

 MYER CENTER 

 FIELD ELEMENT CECOM 

 FT MONMOUTH NJ 07703-5601 

 5 CHIEF 

 (CDs JOINT IED DEFEAT ORGANIZATION 

 only) R KEESEE SES 

 5000 ARMY PENTAGON 

 WASHINGTON DC 20310-5000 

 1 JOINT IED DEFEAT ORGANIZATION 

 (CD S BURNETT 

 only) 5000 ARMY PENTAGON 

 WASHINGTON DC 20310-5000 

ABERDEEN PROVING GROUND

 1 ARMY RSCH LABORATORY - HRED 

 AMSRD ARL HR MD 

 WEAPONS BRANCH REDSTONE 

 T HADUCH 

 BLDG 459 RM 231 

 APG MD 21005