United States Research Institute

Special Report 52

August 2002

Selection For Leadership, Transforming NCO Promotion
U.S. Army Research Institute

i

User Community Report

ARI Special Report 52

Roy C. Campbell

Deirdre J. Knapp

Human Resources Research Organization

In Coordination with:

Tonia S. Heffner

Senior Researcher

August 2002

Selection For Leadership

Transforming NCO Promotion

U.S. Army Research Institute

for the Behavioral and Social Sciences

Internet Site: www.ari.army.mil

Disclaimer: The views, opinions, and/or findings contained in this report are solely those of the authors and

should not be construed as an official Department of the Army or Department of Defense position,

policy, or decision, unless so designated by other documentation. 
Selection For Leadership

Transforming NCO Promotion

ii

This page is blank. Go to the next page.
U.S. Army Research Institute

iii

User Community Report

NCO Vision

An NCO Corps, grounded in heritage, 

values and tradition, that embodies the 

warrior ethos; values perpetual learning; 

and is capable of leading, training, and 

motivating soldiers.

We must always be an NCO Corps that:

• Leads by Example

• Trains from Experience

• Maintains and Enforces Standards

• Takes care of Soldiers

• Adapts to a Changing World

Foreword

The U.S. Army Research Institute for the Behavioral and Social Sciences (ARI) conducts research to support 

Army personnel and training goals. In recognition of the dramatic changes Objective Force operations will 

require, ARI has developed a research program to identify, describe, and address future personnel requirements. 

This report describes one aspect of the program.

The objective of the “Maximizing 21st Century Non-Commissioned Officer Performance” (NCO21) research 

project was to improve 21st century NCO performance by providing the capability to identify and promote 

highly qualified soldiers with exceptional leadership potential. The project developed, validated, and evaluated 

future-oriented job performance predictor and performance measures. The project was conducted with 

support from the Army Deputy Chief of Staff for Personnel (the Army G-1) and the Army Training and 

Doctrine Command (TRADOC). The findings have been briefed to the U.S. Army Total Personnel Command, 

TRADOC, and the Army G-1. This report provides a description of the research effort and discusses how 

the research findings could provide a resource for improving the junior NCO promotion system. 

Zita M. Simutis

Acting Director
Selection For Leadership

Transforming NCO Promotion

iv

Acknowledgements

This report represents an effort of over 4 years duration that had many, many contributors. Only a few are 

acknowledged here. From the Army G-1, Sergeant Major Gerald Purcell provided personnel expertise, 

wise counsel, encouragement, and support when the project reached its crucial latest phase. From the U.S. 

Army Research Institute for the Behavioral and Social Sciences, Dr. Michel Rumsey and Dr. Tonia Heffner 

provided oversight, direction, review, patience, and incomparable professional know-how throughout the 

project. From HumRRO, Ms. Pam O’Quinn provided the design, layout, and graphics for this report. And 

foremost are the almost 3,000 specialists and NCOs who, through their participation as examinees, raters, 

and administrators in the field validation phase, provided the data on which this report is based. It is in their 

interests that projects such as this are undertaken.
U.S. Army Research Institute

v

User Community Report

Contents

The System ....................................................................1

How it Works Now .........................................................3

Objective Force Key Leader Characteristics .................4

Building and Evaluating New System Components ......8

Tools to Measure Job Performance ...............................9

Tools to Predict Performance in the Next Grade .........10

Summary of the Findings .............................................14

Other Possible Applications..........................................15

The Future of NCO Selection and Development..........16

Your Input Counts.........................................................17
Selection For Leadership

Transforming NCO Promotion

vi

This page is blank. Go to the next page.
U.S. Army Research Institute

1

User Community Report

The System 

It has over 81,000 ‘members.’ It moves 30,000 people from soldier to 

sergeant each year. It requires the judgment, perception, and decision 

making skills of the Company Commander and the First Sergeant in a 

procedure that affects not only individuals’ careers but the future welfare 

of the entire Army. It is at once exciting, challenging, rewarding, complex, 

and worrisome. “It” is the semi-centralized promotion system. 

In 1998 the U.S. Army Research Institute for the Behavioral and Social 

Sciences (ARI) started a research project to identify the characteristics 

of 21st century Army NCOs, called NCO21. This effort involved a 

comprehensive analysis of future conditions and job demands to identify 

the knowledges, skills, and attributes (KSAs) that Objective Force NCOs 

will need. Over time, this effort became focused on junior NCOs – the 

movement of specialists/corporals to sergeants and the advancement of 

sergeants to staff sergeants. 

The illustration on page 2 outlines the scope of the NCO21 project. 

The challenge was to develop tools to measure relevant KSAs along 

with tools that measure job performance, then to find out if these 

measures perform as anticipated by administering them to soldiers and 

analyzing the data. If they do work, they would provide an improved 

way of selecting junior leaders. A key element to the project is its future 

orientation – the effort begins to lay a foundation for NCO promotion 

as the Army adjusts to Objective Force operations, allowing the Army 

to be proactive in planning for these changes. Transformation to the 

Objective Force is not served by relying on 20th century promotion 

recommendation and selection procedures. 

The development effort continues. This report is designed to inform and 

update the user community and others who are involved in the promotion 

of those at the entry level of the NCO pyramid.

“ ‘The only lesson of 

history is that men 

never learn anything 

from history,’ said 

George Hegel. In 

the past several 

years promotions to 

sergeant and staff 

sergeant have risen 

dramatically, due in 

part to the Change In 

NCO Structure. Yet 

the question arises, 

‘Are we filling the 

ranks with unskilled 

soldiers,’ and if so, 

are we going to 

pay for it in future 

conflicts?”

CSM Dan Elder, 

The NCO Journal, Winter 2001
Selection For Leadership

Transforming NCO Promotion

2
U.S. Army Research Institute

3

User Community Report

How it Works Now 

The information that currently factors into semi-centralized promotion 

decisions is summarized in the box below. Completing Promotion Point 

Worksheets is currently a significant administrative burden because 

most of the information and the process of pulling it together are not 

automated. The Army is moving quickly toward automation, which will 

make it easier to introduce changes to the system and changes are less 

likely to present an undue burden to the field.
Selection For Leadership

Transforming NCO Promotion

4

Why change the 

promotion system? Didn’t 

it just change? Can’t we 

leave it alone for a while? 

The semi-centralized promotion 

system is never static. Change 

occurs as the overall posture and 

needs of the Army change. Soldier 

evaluation becomes more efficient. 

Job requirements and performance 

predictors become more defined. 

In the near future, the entire semicentralized promotion system will 

become automated, allowing for the 

incorporation of different promotion 

features without additional 

administrative burden. 

But the primary reason is that 

the Army is changing. The NCO 

demands and requirements of the 

Objective Force encompass – and 

then surpass – the demands of 

the 1980s and 1990s. Selection of 

those who lead must reflect these 

changes. 

This does not mean that all of the 

measures and findings outlined in 

this report will become part of that 

future promotion system. Some 

need more work, some are nearer 

term than others, and all need 

official approval before they are 

incorporated. But change to the 

semi-centralized promotion system 

is inevitable and we are developing 

specific ideas for change to consider 

further. 

(In that regard, please note the 

disclaimer that the contents of this 

report are not official Army policy.) 

Objective Force Key 

Leader Characteristics 

The analysis of Objective Force NCO requirements involved two 

objectives: 

4 Developing a description of forecasted conditions affecting 

future NCO performance.

4 Providing a descriptive list of the main qualities (i.e., KSAs) 

needed for effective NCO performance. 

Note that the effort targeted general NCO qualities; not MOS-specific 

requirements. We recognize the criticality of MOS-specific skills, 

particularly as a prerequisite for promotion, but our purview was to 

follow the common soldier model currently used. Parts of specific MOS 

were investigated during the analysis but only if they were typical of 

future trends that all NCOs would need (e.g., digital operations, changing 

missions). 

The analysis identified six primary characteristics that will affect all 

Army personnel in the future. 

Transition to digital operations and an ever increasing pace of 

adaptation of technologies. The hallmark of the early Objective 

Force Army will be the phased introduction of information-age 

operations based on digital connection and supporting software. 

All aspects of combat, combat support, and combat service 

support will be affected. Tactical digital links will be crucial, but 

increasingly non-tactical applications will likely utilize militarized 

adaptations of commercial hardware and software. Change will 

occur rapidly – just keeping up will be a challenge. 

Diverse missions and frequent deployments. While no peer 

opponent is likely to emerge during the first 20 years of the 

century, the number and complexity of Army operations will 

increase. Regional conflicts, peacekeeping, peacemaking, 

humanitarian missions, support to coalition forces, anti-terrorism, 

and domestic assistance are all likely to occur during the era. Each 

non-traditional mission will be somewhat unique and there will 

be little carryover and lessons-learned from preceding missions. 

Deployments will tend to be smaller in size, for shorter periods 

of time (as compared with 20th century conflicts), but more 

frequent.

Diversity of forces and mission-specific organizations. 

Experimentation, change, innovation, and inconsistency will 

mark the period. Some units and soldiers will experience the 

very latest in technology and capability while others will operate 

similarly to the Army of the mid-1990s. More and more units will 
U.S. Army Research Institute

5

User Community Report

“As the Army 

transforms to the 

Objective Force 

we must adjust our 

personnel system 

in anticipation of 

future demands. Our 

focus will always be 

on the development 

and identification of 

a professional NCO 

corps that will meet 

the full spectrum of 

emerging challenges. 

When the promotion 

system changes, it 

will change because 

we have discovered 

proven ways to 

make it better and 

more predictive 

of leadership 

performance.”

SGM Gerald Purcell, 

Directorate of Military 

Personnel Policy, Army G-1

fill niches in equipment, missions, training, and technology and 

there will be less standardization across the force, even between 

like units.

Decentralized operations. Technological advances will 

increase the physical dispersion between elements and between 

supporting and supported units. Employment of elements 

will be less hierarchical. Missions will rely on smaller, more 

autonomous elements. Operations will be driven to lower levels 

with companies and even platoons becoming the focal point for 

many operations.

Changes in training structure, requirements, and delivery. 

Training will be typified by an increasing reliance on technology, 

less institutional attendance, and an increased dependence on selfdevelopment training. Self-development training will become an 

increasing requirement in career learning, including subjects that 

were formerly part of unit training curriculums.

Changes in the youth population and recruiting needs. Army 

active duty non-prior service accession requirements will be 

around 90,000 annually. Competition for high quality youth (i.e., 

high school graduates in the upper two-thirds mental category) 

will stiffen as more choices are available to individuals considered 

in the prime recruiting market.

Based on this operational framework, we defined critical soldier 

requirements (i.e., KSAs). This presented a unique challenge because 

we were projecting KSAs to future requirements that had not, in most 

cases, been fully experienced. In the end, the KSAs are a combination 

of existing, well-established requirements along with some newly 

formed ones. All were redefined to reflect the Army Objective Force 

characteristics. 

The 38 KSAs in the list that starts on page 6 were identified and defined. 

These KSAs are consistent with, and supportive of, the doctrine of 

what leaders must be, know, and do as presented in FM 22-100, Army 

Leadership. Although not designed to fit exactly in the FM 22-100 

framework, the NCO21 KSAs can be roughly sorted into the required 

leadership skills (see chart to the right). For example, Conceptual 

Skills include Adaptability and General Cognitive 

Aptitude. Under Interpersonal Skills would be the 

KSAs of Concern for Soldier Quality of Life and Team 

Leadership.

Both current and future NCO requirements can be 

sorted in different ways, called by different names, and 

identified at different levels of detail. The key is being 

clear on what is expected and building NCO selection, 

training, development, and reward systems around those 

expectations. 
Selection For Leadership

Transforming NCO Promotion

6

Sponsored by... 

This work was directed, and is being 

monitored by, the Directorate of 

Military Personnel Policy (DMPP), 

Army G-1. The Army G-1 is the 

Army’s Human Resource provider, 

responsible for programs, policies, 

and systems that shape and enable 

the manning and well-being of the 

force. It is responsible for enhancing 

the readiness and institutional 

strength of the Army. This work 

supports the Army Transformation 

Program. 

Starred ( ) KSAs were judged by 

subject matter expert panels as 

most important for promotion to E5 

and/or E6. 

NCO21 Knowledges, Skills, and Attributes (KSAs) 

Adaptability. Can modify behavior or plans as necessary to reach goals or to adapt to 

changing goals. Is able to maintain effectiveness when environments, tasks, responsibilities, 

or personnel change. Easily commits to learning new things when the technology, mission, 

or situation requires it. 

Common Task Knowledge and Skill. Possesses the necessary knowledge and skill 

to perform common tasks at the appropriate skill level (e.g., land navigation, field survival 

techniques, and NBC protection). 

Conscientiousness and Dependability. Is trustworthy, reliable, planful, and 

accountable. Respects the chain of command and the value of discipline. Does not try to 

avoid responsibility. 

Directing, Monitoring, and Supervising Work. Works with subordinates one-on-one to 

assign tasks and set individual goals for work and assignments. Ensures that assignments 

are clearly understood. Monitors individual subordinate performance and gives appropriate 

feedback. 

Effort and Initiative. Demonstrates high effort in completing work. Takes independent action 

when necessary. Seeks out and willingly accepts responsibility and additional challenging 

assignments. Persists in carrying out difficult assignments and responsibilities. 

General Cognitive Aptitude. Has the overall capacity to understand and interpret 

information that is being presented, the ability to identify problems and reason abstractly, 

and the capability to learn new things quickly and efficiently. 

Integrity and Discipline. Maintains high ethical standards. Does not succumb to 

pressure to commit prohibited, harmful, or questionable acts. Demonstrates trustworthiness 

and exercises effective self-control. Understands and accepts the basic values of the Army 

and acts accordingly. 

MOS/Occupation-Specific Knowledge and Skill. Possesses the necessary technical 

knowledge and skill to perform MOS/occupation-specific technical tasks at the appropriate 

skill level. Stays informed of the latest developments in one’s particular field. 

Motivating, Leading, and Supporting Subordinates. Recognizes, encourages, and 

rewards effective performance of individual subordinates. Corrects unacceptable conduct. 

Communicates reasons for actions and listens effectively to subordinates one-on-one. 

Fosters loyalty and commitment. 

Need for Achievement. Has a general but not misplaced confidence in own abilities. 

Seeks and enjoys positions of leadership and influence. Is typically enthusiastic and 

energetic. Sets high standards and strives for accomplishment and recognition in almost 

any situation. 

Oral Communication Skill. Speaks in a clear, organized, and logical manner. 

Communicates detailed information, instructions, or questions in an efficient and 

understandable way. Note that this skill refers to how well the individual can speak and 

communicate, not whether technical expertise is high or low. 

Physical Fitness. Meets Army standards for weight, physical fitness, and strength. 

Maintains health and fitness to meet deployablity and field requirements as well as the 

physical demands of the daily job. 

Problem Solving and Decision Making Skill. Reacts to new problem situations 

by applying previous experience and previous education and training appropriately and 

effectively. Does not apply rules or strategies blindly. Assesses costs and benefits of 

alternative solutions and makes timely decisions even with incomplete information. 

Relating to and Supporting Peers. Treats others in the NCO corps in a courteous, 

diplomatic, and tactful manner. Provides help and assistance to other NCOs. Backs up and 

fills in for others when needed. Works effectively as a team member. 

Team Leadership. Communicates team goals and organizes and rewards effective 

teamwork. Leads the team to adapt quickly when missions change and keeps the team 

focused on new goals. Resolves conflicts among team members. Shares relevant information 

with team members. 
U.S. Army Research Institute

7

User Community Report

And, we are.... 

The U.S. Army Research Institute 

for the Behavioral and Social 

Sciences (ARI) is part of the G-1 

function of the Army. Headquartered 

in Alexandria, Virginia, ARI serves 

the Army world-wide with nine field 

locations and a presence wherever 

and whenever there is a personnel 

issue to be addressed. ARI’s 

mission is to maximize individual 

and unit performance and readiness 

to meet the full range of worldwide 

Army missions through advances in 

the behavioral and social sciences. 

It is the primary arm of the Army 

conducting research and analysis 

on personnel performance. ARI’s 

programs support three of the 

Army’s Imperatives: Quality 

People, Leader Development, and 

Training. Learn more about ARI at 

http://www.ari.army.mil.

The Human Resources Research 

Organization (HumRRO) is a 

private, non-profit research firm 

with over 50 years of experience 

and dedication to working with the 

Army. It is staffed with scientists and 

specialists in the areas of military 

policy and functions, personnel 

planning and policy analysis, and 

performance measurement and 

assessment. It is also located 

in Alexandria, Virginia with a 

number of field locations. Find 

out more about HumRRO at http:

//www.humrro.org. 

Together, the ARI-HumRRO team 

takes pride in its approach to 

researching soldier issues. The 

focus is always service to the 

Army. We involve soldiers in our 

projects through panels, surveys, 

interviews, field data collection, 

and product reviews.

NCO21 Knowledges, Skills, and Attributes (KSAs) Continued

Training Others. Evaluates and identifies individual or unit training needs. Institutes formal 

or informal programs to address training needs. Develops others by providing appropriate 

work experiences. Guides and assists subordinates on technical matters.

Adherence to Regulations, Policies, and Procedures. Adheres to policies and follows 

prescribed procedures in carrying out duties and assignments. 

Advanced Computer Skills. Understands numerous computer systems and operating 

systems such as Unix, NT, and Army specific systems and applications. Can perform 

routine troubleshooting of computer systems and applications. 

Basic Electronics Knowledge. Knows general information regarding electronics principles 

and electronics equipment operation and repair. Knows general facts and principles relevant 

for a wide variety of electronics related tasks, but does not necessarily have highly specific 

electronics knowledge required for a particular job. 

Basic Mathematics Facility. Knows and applies addition, subtraction, multiplication, division, 

and simple mathematical formulas. 

Basic Mechanical Knowledge. Knows general information regarding mechanical principles, 

tools, and mechanical equipment operation and repair. Knows general facts and principles 

relevant for a wide variety of tasks that require technical knowledge, but does not necessarily 

have highly specific mechanical knowledge required for a particular job. 

Battlefield Function Coordination and Management. Can individually apply and effectively 

integrate multiple battlefield functions such as direct and indirect fires, communications, 

intelligence, and combat service support to achieve tactical goals.

Concern for Soldier Quality of Life. Is aware of subordinates’ off-duty needs and constraints. 

Is sensitive to others’ priorities, interests, and values, and tries to assist subordinates in 

making their personal and family life better. 

Cultural Tolerance. Demonstrates tolerance and understanding of individuals of other cultural 

and social backgrounds, both in the context of the diversity of U.S. Army personnel and in 

interactions with foreign nationals during deployments or when training for deployment. 

Emotional Stability. Has the tendency to act rationally, to display a generally calm and 

even mood. Generally maintains composure and does not become overly distraught by 

stressful situations. 

General Self-Management Skill. Uses appropriate strategies to self-manage the full range 

of one’s work and non-work responsibilities (e.g., assignments, personal finances, family). 

Such strategies include setting both long-and short-term goals, prioritizing goals, allocation 

of effort and personal resources to goal priorities, and assessing one’s own performance. 

Works effectively without direct supervision, but seeks help and advice from others when 

appropriate. 

Information Management. Effectively monitors, interprets, and redistributes digital display 

information (as well as printed and orally delivered information) from multiple sources to 

multiple recipients. Sorts, classifies, combines, excludes, and presents information so that 

it is usable by others. Does not readily succumb to information overload. 

Knowledge of Inter-Relatedness of Units. Is capable of analyzing how goals and operations 

of own unit are interrelated with other units and systems, and how one unit’s actions affect 

the performance of other units. Can see the larger strategic picture and interpret how one’s 

own unit relates to it. 

Military Presence. Presents a positive and professional image of self and the Army even 

when off duty. Maintains proper military appearance. 

Models Effective Performance. Acts in ways that consistently serves as a model for 

what effective performance should be like, be it technical performance, military bearing, 

commitment to the Army, support for the Army mission, or performance under stressful or 

adverse conditions. Can consistently set an example for others to follow.
Selection For Leadership

Transforming NCO Promotion

8

The Promotion Predictor 

Challenge: 

Promotion is not a reward for past 

performance – it is an expectation 

of future performance. But it is 

also true that the best predictor 

of future performance is past 

performance. Therefore the 

NCO21 KSA list includes job 

performance requirements that are 

applicable (to varying degrees) to 

performance at grades E4, E5, and 

E6. This allows us to consider using 

current performance as a specialist 

(for example) to help predict future 

performance as a sergeant. The 

more time in service, the more 

historical information there is to 

evaluate, but there will always be 

an amount of uncertainty involved. 

Reducing that uncertainty improves 

the promotion system. 

NCO21 Knowledges, Skills, and Attributes (KSAs) Continued

Perceptual Speed and Accuracy. Has the ability to recognize and interpret visual information 

quickly and accurately, particularly with regard to comparing similarities and differences 

among words, numbers, objects, or patterns, when presented simultaneously or one after 

the other. 

Psychomotor Aptitude. Has the ability to coordinate the simultaneous movements of one’s 

limbs (arms, legs), to operate single controls or to operate multiple controls simultaneously, 

and to make precise control adjustments that involve eye-hand coordination. 

Safety Consciousness. Follows safety guidelines and instructions. Checks the behavior 

of others to ensure compliance. 

Self-Directed Learning Skill. Has a clear goal of maintaining continuous learning and 

training over one’s entire career. Is proficient at determining personal training needs, planning 

education and training experiences to meet them, and evaluating one’s own training success. 

Uses efficient personal learning strategies (e.g., identifying specific learning goals, organizing 

the material to be learned, and practicing the new skills in an appropriate context).

Selfless Service Orientation. Commits to the greater good of the team or group. Puts 

organizational goals ahead of individual goals as required. 

Spatial Relations Aptitude. Has the ability to mentally visualize the relative positions of 

objects in two dimensional or three-dimensional space, and how they will be positioned if 

they are moved or rotated in different ways. 

Working Memory. Has the ability to maintain information in memory for short periods of 

time and to retrieve it accurately.

Writing Skill. Communicates thoughts, ideas, and information successfully to others through 

writing. Uses proper sentence structure including grammar, spelling, capitalization, and 

punctuation.

Building and Evaluating 

New System Components 

The identification of important NCO KSAs was only the first step. 

Identifying KSAs has little practical application unless we measure 

them so they can be used for making decisions. Measurement presents 

its own challenges. There are no perfect ways to measure characteristics 

of people that, unlike height and weight, are not directly observable. Our 

goal is to make the measurement of soldiers’ capabilities as accurate 

and reliable as possible – removing irrelevant factors and making the 

selection fair and objective. This usually involves standardizing how 

soldiers are evaluated. 

We developed several potential measures that could be used as a basis 

for making decisions about promotion. These became our predictor

measures because we use them to predict NCO performance at the 

next pay grade. We also developed two separate measures of NCO job 

performance – one for current performance and another for performance 

under future conditions. These measures were the criteria against 

which the predictor measures were evaluated. Both the predictor and 

job performance measures went through extensive, painstaking, and 
U.S. Army Research Institute

9

User Community Report

time consuming development procedures. Then the predictor measures 

were administered to hundreds of soldiers and performance ratings were 

obtained from their supervisors in a field validation.

Our research focused on the predictability of a soldier’s performance in the 

next highest grade level as determined by the relationship between the KSA 

(predictor) measures and the supervisor ratings. Rather than track soldiers 

over time, we administered the predictor and job performance measures to 

sergeants and staff sergeants. Looking at the relationship between predictor 

and job performance scores for the E5 soldiers tells us what is likely to 

work best for promoting specialists/corporals to sergeant and looking at 

the same data for the E6 soldiers tells us what predictors will likely work 

best for promoting sergeants to staff sergeants. If a predictor measure has 

a high correlation with supervisors’ job performance ratings, that means 

it could be useful for predicting performance. An even better outcome 

would be if results for the experimental predictor tools, combined with the 

current Promotion Point Worksheet, show a significantly higher correlation 

with the job performance ratings than does the current system by itself. 

Adding such tools to the current system would improve the accuracy of 

promotion decisions. 

Tools to Measure Job 

Performance 

Within the military, supervisor ratings have been well established as an 

accurate means of evaluating both performance and potential of subordinate 

soldiers. It is a comfortable role for most supervisors. Two different 

supervisor rating instruments were developed: (a) one set to evaluate 

performance in the current Army, and (b) another set to anticipate how 

soldiers will perform under expected future Army conditions. 

Observed Performance Rating Scales 

These scales were designed to provide an overall picture of the individual’s 

current performance, plus potential as a senior (E7-E9) NCO. NCO 

performance is multi-dimensional and we identified 19 areas of functional 
Selection For Leadership

Transforming NCO Promotion

10

Observed Performance Rating Scales – NCO Areas of 

Performance 

n MOS/Occupation-Specific Knowledge and Skill

n Common Task Knowledge and Skill

n Computer Skills

n Writing Skill

n Oral Communication Skill

n Level of Effort and Initiative on the Job

n Adaptability

n Self-Management and Self-Directed Learning Skill

n Demonstrated Integrity, Discipline, and Adherence to 

Army Procedures

n Acting as a Role Model

n Relating to and Supporting Peers

n Cultural Tolerance

n Selfless Service orientation

n Leadership Skills

n Concern for Soldier Quality of Life

n Training Others

n Coordination of Multiple Units and Battlefield 

Functions

n Problem Solving/Decision Making Skill

n Information Management 

For a complete set of the Observed Performance Rating Scales 

and other selected NCO21 measures, please submit a request to 

NCO21@ari.army.mil. 

requirements. A sample scale is shown on page 9 and the complete list 

of areas is shown in the box below. Supervisors provide scaled ratings 

based on what they have observed about their soldiers’ performance. 

This provides a complete description of the individual, identifying both 

strong and weak areas. The results are used as one set of criteria to 

evaluate the predictor measures.

Expected Future Performance Rating Scales 

It was necessary to develop an instrument that focused on performance 

under future conditions because the focus of the work is to improve 

NCO promotion in the future Army. This instrument was based on the 

analysis of emerging Objective Force requirements and conditions, 

largely reflected in the six characteristics described on pages 4-5. 

Collectively, these scales provide a picture of how supervisors predict 

soldiers will perform in this future environment. Taken together with the 

Observed Performance Rating Scales, these scales provide the complete 

set of criteria for evaluating the predictor measures. 

Tools to Predict 

Performance in 

the Next Grade 

Predictor measures were developed to 

assess whether a soldier possesses the 

relevant KSAs for promotion to a higher 

level of leadership. The predictor measures 

were designed to be suitable for potential 

inclusion in the Army’s semi-centralized 

NCO promotion system, which means we 

had to consider the realities of operational 

administration. The goal is to provide 

meaningful information for making 

promotion decisions from the grades of 

E4 to E5 and from E5 to E6. 

The Personnel File Self-Report

Description: The Personnel File Self-Report 

recreates the administrative information 

found on the Promotion Point Worksheet. 

During the field validation it was not 

possible to access personnel records and 

extract Promotion Point information. 

However, it was necessary to analyze the 

administrative point areas currently used 

– this, in effect, was baseline information 

reflecting the promotion status quo. 
U.S. Army Research Institute

11

User Community Report

Previous work has indicated that soldiers are generally pretty accurate 

in providing information about themselves. We therefore asked soldiers 

for the information normally found on the Promotion Point Worksheet. 

The self-report form was not meant to mirror exactly the Promotion 

Point Worksheet, only approximate it. The Personnel File Self-Report 

information was sufficiently detailed so that we were able to simulate 

an Administrative Points score for each soldier. 

Field Validation Results: The simulated Administrative Points promotion 

score predicted performance fairly well. We found that some of the 

administrative categories predicted how soldiers would perform as a 

sergeant better than how they would perform as a staff sergeant, and 

vice versa. We also looked at these areas by job categories (clusters 

of MOS) and found some differences. While we must be cautious in 

this regard because our sample did not include all MOS or represent 

all career management fields, there is evidence that prediction could 

be improved by weighting the administrative categories differently by 

pay grade and MOS or MOS cluster. Indeed, the experimental predictor 

measures described below all showed differences in how they worked 

by pay grade and MOS type. 

Armed Services Vocational Aptitude Battery (ASVAB)

Description: The ASVAB is an existing measure used for both selection 

into the Army and for MOS classification. Additionally, ASVAB scores 

are used after a soldier is in service to determine reenlistment, and 

sometimes, schooling and duty options. Soldiers can retake the ASVAB 

while in service – then it is called the Armed Forces Classification Test 

(AFCT). 

The ASVAB has several subtests that are combined to create various 

composite scores. One composite – the Armed Forces Qualification Test 

(AFQT) – is used to identify high quality soldiers for selection into the 

Army. Nine composites are used as prerequisites for classification into 

specific Army jobs. The 11th composite – General Technical (GT) – is 

used extensively in-service for schooling and reenlistment options. The 

ASVAB provides a direct measure of the soldier’s general cognitive 

aptitude which is an underlying requirement to many aspects of NCO 

performance such as problem solving and decision making. 

Field Validation Results: The ASVAB showed some capacity to improve the 

prediction for both E5 and for E6 performance over the current system. 

In other words, it would improve slightly over what is currently being 

used in the Promotion Point Worksheet Administrative Points areas. 

Temperament Indicator Scales

Description: These scales provide measures of a variety of life skills, 

reactions to life events, behaviors, and psychological constructs. They 

are directly related to such attributes as adaptability, dependability, and 

interpersonal skills. There is a strong leadership component to these
Selection For Leadership

Transforming NCO Promotion

12

Faking It?

A concern in measure development 

is that there are certain instruments 

where soldiers, if they have a stake 

in the outcome (such as promotion), 

will give the input they think the Army 

expects to hear, not necessarily the 

most truthful response. Some of the 

trial predictor measures are more 

susceptible to this than others. For 

example, in some measures (the 

ASVAB for example) there is a 

“right” answer, so faking is not an 

issue (though cheating can be). But 

in other measures, the answer is not 

right or wrong but the answer tells 

us something about the person. If 

people are asked how much they 

cheat on their income tax, they may 

be less than truthful if the response 

can be identified with them. In that 

example, people can tell the “good” 

answer from the “bad” answer and 

can vary their response based on 

what they feel the effect of providing 

a completely truthful response might 

be.

There are ways in which the 

instruments can be designed to 

help guard against the inclination. 

For example, some of the 

Temperament Indicator scales use 

a so-called “forced choice” strategy. 

There are also procedures that can 

be used to identify respondents who 

are probably faking. 

Concern about this issue is a 

serious requirement. It is essential 

that a trial validation period under 

operational conditions be part of the 

development. The Army (its soldiers 

and leaders) must have confidence 

that potential promotion instruments 

are fair, accurate, and reliable.

scales that has been established in a variety of other Army applications. 

Results show that these scales can be used to predict far ranging 

behaviors from individual physical fitness to personal integrity. 

Field Validation Results: Some of the individual behavioral areas 

assessed by these scales suggest that they could substantially improve 

the prediction of performance over the current Administrative Points. 

For example, Social Maturity and Tolerance for Ambiguity scores 

showed promise for predicting performance for sergeants whereas the 

Adjustment score was particularly useful for predicting staff sergeant 

job performance.

Experience and Activities Record (ExAct) 

Description: The ExAct assesses how often a soldier has engaged in 

specific activities or experiences that may predict performance at the 

next grade. The ExAct elicits verifiable information about individual 

experiences that are often considered by supervisors when evaluating 

personnel for promotion. It is not meant to capture normal duty or MOS 

performance but to identify those experiences and activities that occur 

beyond the expected performance for the job and grade. It codifies what 

is already being done on an informal basis. 

Field Validation Results: The ExAct did a fairly good job at predicting 

performance for sergeants. However, we found it provided only limited 

improvement to the prediction of performance over the Administrative 

Points portion of the current Promotion Point Worksheet. 

The Leadership Judgment Exercise (LeadEx)

Description: The LeadEx assesses the effectiveness of judgments about 

appropriate courses of action in various job-related, supervisory-based 

scenarios. Situations address areas of problem solving and decision 

making, team leadership, dealing with subordinates, and conducting 

training. 
U.S. Army Research Institute

13

User Community Report

Although the LeadEx items 

look like traditional multiplechoice test questions, all 

responses have some degree 

of acceptability; that is, there 

is not a single “correct” 

answer with all other choices 

being incorrect. Because each 

possible response has been 

rated by senior NCOs as to its 

effectiveness for handling the 

problem presented, the scoring 

system allows some fairly 

precise rankings of supervisory 

effectiveness. 

Field Validation Results: 

The LeadEx showed itself to 

be a promising instrument, 

particularly for predicting 

performance for sergeants. 

It significantly improved the 

prediction of performance over 

and above current promotion 

requirements.

Semi-Structured Interview 

Description: The semi-structured interview was designed to provide a 

board evaluation but under more controlled conditions than exists with 

the current promotion board process. The interview is structured to 

measure specific leadership traits. It provides for questions with in-depth 

response requirements based both on the individual’s past experiences 

and on hypothetical situations. 

Soldiers are evaluated 

using a structured rating 

procedure.The interview 

provides standardized 

measurement of adaptability, 

effort and initiative, selfmanagement and self-directed 

learning, and leadership 

potential. 

Field Validation Results: During 

the field validation, the semistructured interview showed a 

strong validity for predicting 

performance for E5 soldiers. 
Selection For Leadership

Transforming NCO Promotion

14

1 Validity estimates are based on correlations between the predictors and criterion . These values 

can vary from 0 to 1, where 0 indicates no association between predictors and criterion, and 1 

indicates a perfect association. Thus, the higher the values, the better the prediction. These values 

are based on estimates that have been corrected for standard statistical artifacts (i.e., criterion 

unreliability, range restriction in the predictor, and shrinkage).

Because of time constraints, the interview was not administered to the 

E6 soldiers so no conclusions were reached about this group.

Summary of the Findings

Validation is finding out what works, what doesn’t work, what works 

a little bit, and what works a lot. In this case, with a lot of different 

measures and a lot of different groups (E4s, E5s, E6s), we need to be 

cautious about what we conclude. Study, analysis, and review of the 

data that we gathered will continue, but we do have some preliminary 

indications about the current promotion system and what effect changes 

would have. 

Just how much improvement is achievable over the current system 

cannot yet be precisely quantified. As indicated in our discussion of 

each individual measure, however, most of the predictors that we 

examined showed potential as additions for enhancing the Army’s 

semi-centralized promotion system. In most cases, the results were 

stronger when considering promotion to E5 than when applied to E6 

promotion.

The figures on pages 14-15 indicate an approximate improvement if all

of the experimental predictor measures developed in the NCO21 project 

were added to the existing Promotion Point Worksheet1.

Taken together, the findings 

indicate that the semi-centralized 

promotion system could be 

improved and that elements of 

the measures we studied are 

good candidates to be part of that 

improvement. There is also some 

compelling evidence that there are 

differences between how various 

types of information can best be 

used to forecast performance at 

the sergeant and staff sergeant 

levels. More accurate promotion 

decisions would be possible if 

these differences were reflected 

in the system (e.g., by varying 

the number of points assigned to 

different pieces of information on 

the Promotion Point Worksheet, 
U.S. Army Research Institute

15

User Community Report

depending on whether one is up 

for an E5 or E6 promotion). This 

would be easy to implement in 

an automated promotion system. 

Similar differences can be 

identified by types of jobs (MOS 

or clusters of MOS) but the field 

validation is not broad enough 

to draw more than preliminary 

indications that there might be 

some effect that would warrant 

different promotion treatment 

by MOS.

Other 

Possible 

Applications

During the field validation, a lot of reaction, some pro and of course, 

some con, to the different measures was provided by the participants 

and supervisors. However there were three that consistently generated 

the most interest for potential uses other than the semi-centralized 

promotion structure. 

Observed Performance Rating Scales: These were seen as a 

valuable counseling tool by many of the supervisors. Mentioned most 

frequently as a feature was the behavioral anchor descriptions that 

could be used (as shown in the example on page 9), not only to define 

current levels of performance but also to pinpoint what is needed for 

individuals to get to higher levels. Participants pointed out that the scales 

were a handy working document that could be shared with subordinates. 

Several supervisors highlighted 

the flexibility of the scales – that 

they could be tailored for each 

individual, concentrating on just 

a few of the areas at a time. To 

get a broader perspective, ratings 

could be collected from peers 

and subordinates, as well as 

supervisors.

Experiences and Activities 

Record (ExAct): This too was 

seen as a counseling tool but 

primarily as a roadmap for those 

seeking to improve themselves. 

Several supervisors thought of it as 

being particularly helpful to use 
Selection For Leadership

Transforming NCO Promotion

16

NCOES Applications...

It is not necessary for individual 

measures that are proven 

predictors to be integrated into 

the semi-centralized promotion 

system to have an effect on future 

leadership. Many of the measures 

can be used as counseling tools 

to change or guide behaviors, not 

just to measure KSAs. The path to 

NCO leadership is one of learning 

and experience in a great number of 

areas. Many of these measures can 

provide an individualized roadmap 

for soldiers.

A logical place to use some of these 

measures is in NCOES – PLDC 

and BNCOC. NCOES is currently 

undergoing study and evaluation as 

part of the Army Transformation and 

we will make our findings available 

to those charged with this mission.

with their “fast track” specialists and corporals. Not everyone agreed 

completely with all of experiences and activities listed and some felt 

they would modify or delete some of the items. Foremost among the 

changes was to add blank lines to list MOS- or unit-specific experiences 

and activities. 

Semi-Structured Interview: Most of the reaction came from senior 

NCOs who served as board participants during the field validation. 

Almost all had prior experience as board members. They commented 

that there were a lot of board activities in the Army besides the semicentralized board, including E4 boards, soldier of the month boards, 

Sergeant Morales, and Audie Murphy boards. Since little guidance 

exists for preparing for and interacting with the soldier during board 

proceedings, they felt that the semi-structured interview would be a 

good training vehicle for supervisors, giving them specific practice in 

how to apply and interpret results.

The Future of NCO 

Selection and 

Development

Significant changes to the semi-centralized promotion system will 

be deliberate, based on scientific evidence of improvement, and in 

conjunction with other NCO leadership initiatives, coupled with 

command and user buy-in. This effort is a first step to lay a foundation 

for inevitable change to the system. It has identified some viable tools 

that could improve performance of the NCO corps either through 

changes to promotion requirements or through counseling or NCOES 

improvements.

An option that will be evaluated further is to combine elements of some 

of the promising NCO21 measures (e.g., the LeadEx and Temperament 

Indicator) to form a Leadership Assessment Tool that could be 

administered to semi-centralized promotion candidates. A logical next 

step is to try-out such a concept under operational conditions over a 

period of time in an experimental setting that could last for 1-2 years. 

It would be “experimental” in that soldier’s promotions would not 

be affected by the new leadership measure. Other than that, it would 

be applied along side the current promotion criteria on a wide basis 

to soldiers under existing field conditions. Work is progressing on 

implementing such a try-out.
U.S. Army Research Institute

17

User Community Report

Measuring performance...

In this study, we used supervisor 

ratings of observed and expected 

future performance as the criteria 

for evaluating the predictors. 

Performance ratings are measures 

with established effectiveness and 

are relatively easy to collect. But 

they are not the only measures 

that could be used. MOS-specific 

tactical and technical knowledge 

tests and hands-on performance 

tests of technical and leadership 

skills would also be valuable 

measures. The drawback to these 

is their difficulty in preparation and 

administration. However, we are 

exploring ways that simulations 

can be used to provide standard 

evaluations and reduce the 

overhead requirements of these 

measures. Further, ARI is currently 

conducting research to develop 

MOS-specific tactical and technical 

competency tests that overcome 

the preparation, administration, 

and operational obstacles of former 

MOS skill tests. 

[The Army must]... “develop Enlisted Personnel 

and Leader Development Systems that produce a 

deployable, agile, versatile, lethal and survivable 

soldier who is persuasive in peace, invincible 

in war, and victorious in any form of conflict.”

Excerpt from the Army Developmental Systems XXI vision statement

Your Input Counts

Now you can provide feedback by giving us your reaction to what we 

have done and what we propose to do. Any comments are welcome. 

Send your input to:

Directorate of Military Personnel Policy (DMPP)

Army G-1

DAPE-MPE-PD

300 Army Pentagon

Washington, DC 20310-0300

DSN: 224-7182 or COMM: 703-614-7182

Or provide input by email to

gerald.purcell@hqda.army.mil

Back cover page.

United States Army Research Institute for the Behavioral and Social Sciences

www.ari.army.mil

End of document.