Method

Sample

All available CPR ratings of USMA cadets were obtained and
analyzed for three consecutive academic terms during 1992 and
1993. Although sample sizes increased substantially over time,
the research procedures followed and the results obtained were
highly similar across terms. For this reason, our report
addresses only research related to the Fall semester of 1993, for
which ratings of over 2,000 different cadets were received.
Because the exact nature of this sample is difficult to
understand without information on the measurement procedures
involved, additional information on the sample is provided in the
Procedure section on pages 11 and 12.

Measures

The Leadership Evaluation and Developmental Ratings (LEADR)
system at USMA incorporates both evaluative and developmental
processes, as its name implies. The leadership grade is the main
evaluative measure in the LEADR system, and represents the
official evaluation of a cadets overall performance ina
leadership role for a specified academic term or summer detail.
The CPR is employed as the primary developmental instrument in
LEADR. It supplies a common framework for observing and rating
cadet performance at the Academy. Its primary function is to
provide feedback and coaching to cadets on their leadership
performance in accordance with duty position® and cadet class
requirements (U.S. Corps of Cadets, 1995).

Leadership grade. A leadership grade is given to each cadet
using a conventional 5-point letter to numeric conversion: A (4),

B (3) C (2), D (1), and F (0). A forced distribution system is
used to limit the number of cadets receiving grades of A, B, or C
(U.S. Corps of Cadets, 1995). The result of this forced

distribution system is that no more than 20% of the cadets within
a grader's span of control can receive an A, no more than 40% can
receive a B, and no more than 40% can receive a Cc. The awarding
of marginal (D) and unsatisfactory (F) grades is not governed by
this forced distribution system, as they are awarded on a
relatively infrequent basis, usually only to those cadets that
fail to meet USMA standards of performance in their assigned duty
position. Although an unofficial grade is given to each cadet

ta duty position is a specific job assigned to a cadet each
term, varying in terms of both its functional task requirements
and its level of supervisory responsibility. Assignments to duty
positions are based upon a cadet's graduation year (class) and
their performance in prior positions.



military bearing. Focusing on the interrelationships existing
among the CPR dimensions, however, the principal components
analyses implied that raters kept up to four different concepts
in mind as they rated cadets on the dimensions. Of these four
concepts, only one was related to duty motivation and military
bearing, and it tended to be least important in explaining the
variation among dimension scores.

It is unclear exactly why the two types of analyses yielded
such different results. One explanation is that raters tend to
base their holistic judgments about leadership performance on
those things that they see most often or most easily. Because
self-management concepts like duty motivation and military
bearing are observable in a variety of situations (OIR, 1989),
holistic judgments can be formed by the sheer volume of
behavioral incidents occurring in those dimensions. It is also
possible that those same dimensions are used most often by raters
who had the least opportunity to observe cadets during the rating
period. For instance, one could easily rate cadets on duty
motivation and military bearing, without having observed them
interacting with subordinates. Secondarily, raters can also base
their judgments on those things they value or find most important
in a particular situation. Regarding CPR global scores provided
by subordinate raters, for example, this secondary consideration
may help to explain why the developing subordinates and
influencing others dimensions each accounted for more variance
than the military bearing dimension, which is probably more
observable.

Even if most raters base their holistic judgments on the
above considerations, they still appear quite capable of
differentiating cadet performance using most CPR dimensions.
Instances where they cannot tend to have reasonable explanations
(e.g., cadets in the Fourth Class are rarely rated on the
delegating, supervising, and developing subordinates dimensions
because they have no subordinates). One indicator of the
construct validity of a leadership behavior rating system is the
extent to which raters assign similar scores to those dimensions
that are the most similar conceptually. Although our proposed
four-component structure appears to have a plausible conceptual
foundation in this regard, it only becomes useful when it helps
to formulate accurate predictions about CPR ratings that are then
confirmed through future research. Though not reported here, we
should note that the same four-component structure emerged in two
earlier investigations involving smaller cadet samples.

The analysis of CPR dimension use by different types of
raters provided additional information about the importance of
the duty motivation and military bearing dimensions. Generally,
these dimensions were the ones most often used by raters to
describe a cadet's relative strengths and weaknesses. However,
there were some consistent differences found in the ways the
various types of raters used the remaining dimensions. Superiors
and subordinates appeared to have the most distinctly different

30



Table 9

Rank Order of CPR Dimension Use by Superior, Peer, Subordinate,
and Instructor Raters

Type of Rater

Dimension Superior Peer Subordinate Instructor
Duty Motivation 1 (90) 1 (71) 1 (61) 1 (50)
Military Bearing 2 (81) 2 (55) 2 (50) 4 (31)
Teamwork 5 (56) 3 (45) 6 (28) 5 (23)
Influencing

Others 6 (38) 7 (30) 3 (41) 8 (14)
Consideration

for Others 9 (27) 5 (36) 5 (34) 7 (18)
Professional

Ethics 11 (20) 8 (18) 10 (18) 9 (14)
Planning and

Organizing 3 (68) 4 (39) 8 (26) 3 (37)
Delegating ~ 12 (19) 10 (15) 9 (19) 11 (5)
Supervising 8 (28) 12 (11) 11 (18) 12 (5)
Developing

Subordinates 7 (34) 9 (18) 4 (35) 10 (5)
Decision Making 10 (24) 11 (15) 12 (10) 6 (21)

Oral and Written
Communication 4 (57) 6 (33) 7 (26) 2 (42)

Note. Depending on the dimension, there were between 2013 and
2375 cadets rated by superiors, between 2042 and 2238 rated by
peers, between 1221 and 1294 rated by subordinates, and between
1367 and 2524 rated by instructors. Approximate dimension use
percentages are shown in parentheses. Dimension usage was
defined as the percentage of cadets having a particular dimension
rated as a relative strength plus the percentage having that same
dimension rated as a relative weakness.

26



Hollander, E.P. (1965). Validity of peer nominations in
predicting a distant performance criterion. Journal of
Applied Psychology, 49, 434-438.

Kanungo, R.N., & Misra, S. (1992). Managerial resourcefulness: A
reconceptualization of management skills. Human Relations,
45, 1311-1332.

Korotkin, A.L., Mumford, M.D., Yarkin-Levin, K., Wallis, M.R., &
Fleishman, E.A. (1986). Taxonomic efforts in the description
of leadership behavior: A general approach (Research Note
86-21). Alexandria, VA: U.S. Army Research Institute for the
Behavioral and Social Sciences. (AD A169 193)

McEvoy, G.M., & Beatty, R.W. (1989). Assessment centers and
subordinate appraisals of managers: A seven-year examination
of predictive validity. Personnel Psychology, 42, 37-52.

Moore, J.V., & Smith, R.G., Jr. (1952). Aspects of
noncommissioned officer leadership (HRRC Technical Report
52-3). San Antonio, TX: Human Resources Research Center.

Morgan, R.B. (1989). Reliability and validity of a factor
analytically derived measure of leadership behavior and

characteristics. Educational and Psychological Measurement,
49, 911-919.

Mount, M.K. (1984). Supervisor, self- and subordinate ratings of
performance and satisfaction with supervision. Journal of
Management, 10, 305-320.

Mumford, M.D. (1986). Leadership in the organizational context: A
conceptual approach and its applications. Journal of Applied
Social Psychology, 16, 508-531.

Office of Institutional Research (1989). Evaluation of the

sequential leader development system: Content validity. West
Point, NY: U.S. Military Academy.

Rogers, R.W., Lilley, L.W., Wellins, R.S., Fischl, M.A., & Burke,
W.P. (1982). Development of the precommissioning leadership
assessment program (ARI Technical Report 560). Alexandria,
VA: U.S. Army Research Institute for the Behavioral and
Social Sciences. (AD A130 927)

Shipper, F., & Wilson, C.L. (1991, July). The impact of
managerial behaviors on group performance, stress, and
commitment. Paper presented at the Impact of Leadership
Conference, Center for Creative Leadership, Colorado
Springs, CO.

36



The sixteen dimensions of the LAP are oral communication,
oral presentation, written communication, initiative,
sensitivity, influence, planning and organizing, delegation,
administrative control, problem analysis, judgment, decisiveness,
followership, technical/tactical competence, physical stamina,
and mission accomplishment (U.S. Army Cadet Command, 1993). The
development of the initial LAP was based on a job analysis of
ROTC precommissioning education and training requirements
(Rogers, Lilley, Wellins, Fischl, & Burke, 1982). In their
analysis, a tentative set of dimensions was derived from a review
of the literature, interviews with job incumbents (i.e., Second
Lieutenants), and critical incident interviews with individuals
who had occupied or supervised individuals in the incumbent
position (i.e., Captains). Following these steps, a
questionnaire was administered to another sample of Captains to
investigate the importance of each dimension for achieving
success in the Second Lieutenant position. A content validity
analysis was employed to determine the final set of dimensions
used in the LAP.

The twelve dimensions of the CPR are duty motivation,
military bearing, teamwork, influencing others, consideration for
others, professional ethics, planning and organizing, delegating,
supervising, developing subordinates, decision making, and oral
and written communication (U.S. Corps of Cadets, 1995). They
were developed from a job analysis conducted by USMA's Office of
Institutional Research (Office of Institutional Research [OIR],
1989). Techniques they employed were similar to the ones used by
the creators of the LAP (Rogers et al., 1982). The twelve
dimensions resulting from the OIR analysis provide a common
framework for observing and rating the performance of USMA cadets
in a wide variety of leadership roles.

Table 1 shows a comparison of the three U.S. Army leadership
classification systems (CAL, LAP, and CPR). It appears that a
substantial amount of comparability exists among the dimensions,
although they are referred to by slightly different terms. There
are a few disparities across the leadership classifications that
may relate to unique responsibilities required for successful
performance in specific leadership situations. One difference is
that the CPR has no dimension involving technical and tactical
proficiency, unlike the CAL and LAP classification systems. This
dimension may not have been included in the CPR because the
technical and tactical competence of USMA cadets is evaluated
separately within the overall USMA curriculum.

In addition, the LAP and CPR possess some common features
not found in the CAL classification system. For example, the
LAP's dimension of initiative and the CPR's dimension of duty
motivation both connote an active performance orientation,
generally thought to be important for successful task
accomplishment and effective leadership. Similarly, the
dimensions of physical stamina in the LAP and military bearing in
the CPR are related to the extent they both involve maintaining



CADET PERFORMANCE REPORT
PART I: ADMINISTRATIVE DATA

a. Name: DOE, JOHN b. Grad Yr: 94 c. AY Company: 14 d. Rank: SGT
e. Duty Assignment: SQD LDR f. Organization: 1ST SQD, 3D PLT, CO K, CBT
g. Report Type: Chain of Command

PART II: AUTHENTICATION

a. Name: Smith, Daniel b. 07/31/93 c. Phone 938-2494
b. Duty Position: PLT SGT e. Organization, LDB

PART III: PERFORMANCE OF DUTIES

a. DUTY MOTIVATION: RATING: 4 FOCUS: s
Always enthusiastic. Very detailed in planning and preparing training.
Excellent use of hip pocket training to reinforce marksmanship principles at
BRM training.

b. MILITARY BEARING: RATING: 4 FOCUS: x
Always completed PT runs and road marches; satisfactory room and personal
appearance. Sometimes gave the impression through facial expressions that he
found it difficult to accept constructive criticism.

c. TEAMWORK: RATING: 2 FOCUS: n
Returned one hour late from leave and was, as a result one hour late for a
squad meeting he scheduled.

d. INFLUENCING OTHERS: RATING: 3 FOCUS: x
e. CONSIDERATION OF OTHERS: RATING: 2 n
Needs to be less critical of peers. Shows low tolerance of those who don't
seem to measure up to his standards. Sometimes demonstrates this in chain of
command meetings.

f. PROFESSIONAL ETHICS: RATING: 3 FOCUS: x
g. PLANNING AND ORGANIZING: RATING: 4 FOCUS: x

Best of 4 sqd ldrs in making good use of time and resources. Showed
flexibility in reacting to last minute changes.

h. DELEGATING: RATING: 4 FOCUS: s
Made excellent use of a prior service New Cadet to assist other New Cadets in
areas such as assembling LBE and disassembly/assembly of M16 rifle.

i. SUPERVISING: RATING 4 FOCUS: x
Above average in training his squad in general military training and providing
them with feedback on their mastery of the skills.

j. DEVELOPING SUBORDINATES: RATING: 4 FOCUS: s
A detailed, thorough trainer. Cdt Doe trained and led by personal example in
training that included: CS Gas Chamber, mountaineering, BRM, and IMT.

k. DECISION MAKING: 3 FOCUS: x
1. ORAL AND WRITTEN COMMUNICATIONS: 3 FOCUS: x

PART IV: OVERALL PERFORMANCE AND POTENTIAL (RATER)
a. Overall Ranking: UPPER 25 PERCENT

b. General comments on performance and overall potential.
Above average squad leader. Thrives on developing subordinates. Above avg
potential.

c. Recommendations for improvement.
Work on interpersonal skills and relationships with peers. Be more
understanding and tolerant of others' shortcomings.



subordinates during their Fourth Class Year. Second, subordinate
ratings of First Class cadets (Class of 1994) were not required.

In terms of the overlap of cadets rated by superiors with
cadets rated by other types of raters, we found that 1267 of the
2375 cadets who had been rated by superiors were also rated by at
least one peer (53%). Similarly, 994 of the cadets rated by
superiors were also rated by one or more subordinates (59% of the
upper three classes), whereas 1461 were rated by one or more
instructors (62%).

To minimize potential confusion in understanding our
analyses and results, readers should keep in mind that our
research samples consisted of the numbers of different cadets
rated by different types of raters. Our reported samples were
not based on either numbers of raters or numbers of ratings.
Thus, our unit of analysis was always an individual cadet ratee.

Table 2

Number of Cadets Rated by Class and Type of CPR Rater

Type of CPR Rater

Class Superior Peer Subordinate Instructor

1994 454 (19) 344 (15) 123 (10) 643 (25)

1995 569 (24) 530 (24) 532 (41) 597 (24)

1996 651 (27) 616 (28) 639 (49) 533 (21)
not

1997 701 (30) 748 (33) applicable 751 (30)

Total 2375 (100) 2238 (100) 1294 (100) 2524 (100)

Note. Approximate class percentages are shown in parentheses for
each type of rater.

13



views of cadet performance. Superiors were more likely to focus
on those dimensions related to the CPR's cognitive component
(i.e., planning and organizing, decision making, and oral and
written communication). Subordinates were more likely to focus
on two dimensions related to the informal interpersonal component
(i.e., influencing others and consideration for others). On the
formal interpersonal component superiors focused on supervising,
while subordinates focused on developing subordinates. Similar
to superiors on some dimensions and similar to subordinates on
others, peers were most distinctive in their relatively greater
emphasis on two other dimensions related to the informal
interpersonal component (i.e., teamwork and professional ethics).
Lastly, instructors tended to have a particularly strong focus on
dimensions related to the cognitive component, even more so than
cadet superiors. Yet, instructors had a relatively weak emphasis
on all dimensions related to the formal interpersonal component.

The results of the analysis of CPR dimension use, coupled
with the regression analyses pertaining to the CPR global score,
are highly consistent with earlier findings in the literature
about the effects of rater position on the leadership evaluation
process (Cann & Siegfried, 1987, 1990; Halpin, 1953; Harris &
Hogan, 1992; Moore & Smith, 1952; Mount, 1984). As they relate
to broader classifications of leadership behavior found in the
literature (Fleishman, 1973; Fleishman et al., 1991; Korotkin et
al., 1986), our findings generally suggest superiors to be more
task-oriented and subordinates to be more person-oriented in
their evaluations of others. Peers seem oriented to both people
and tasks. Though both peers and subordinates had an orientation
toward people, we found peers tended to focus on group relations
(e.g., teamwork) and subordinates tended to focus on individual
relations (e.g., developing subordinates) .

While our report has highlighted the distinct perspectives
of different types of raters, our findings also imply most raters
held superior, peer, and subordinate perspectives of leadership
performance at the same time. Due to the design of USMA's
leadership evaluation system, a cadet can rate one group of other
cadets from a superior perspective, a second group from a peer
perspective, and a third group from a subordinate perspective.
Conversely, a cadet can be judged by others from these three
perspectives as well. Because our investigation focused on
ratees rather than raters, we cannot state with certainty that a
particular rater or group of raters held all three perspectives
simultaneously. Yet, our overall results could not have been
obtained if most cadets held a single perspective. It seems
unlikely that a cadet maintained one exclusive orientation in all
of their superior, peer, and subordinate ratings of others.
Rather, our findings suggest most cadet raters maintained a task
orientation in their superior ratings, both a group relations and
a task orientation in their peer ratings, and an individual
relations orientation in their subordinate ratings.

31



FOREWORD

The Center for Leadership and Organizations Research (CLOR),
jointly established by the U.S. Military Academy (USMA) and the
U.S. Army Research Institute for the Behavioral and Social
Sciences (ARI), conducts programmatic research on Army-wide
priorities in the areas of organizational leadership and leader
education, training, and development. One of the CLOR's major
research efforts is its Leadership Development (LEAD 21) research
program, initiated in 1994 to increase understanding of the
leadership development process. This program involves the
creation of a longitudinal data base, begun with USMA cadets in
the class of 1998, which will be used for describing changes in
the leadership behavior of individual leaders over time, as well
as for identifying those experiences that contribute most to
successful leader development.

A crucial component of any leadership development research
program, particularly longitudinal programs, is an effective
method of measuring leadership behavior. One measure that holds
considerable promise in this regard is the Cadet Performance
Report (CPR), a twelve-dimension leadership behavior rating
system currently used by USMA personnel to evaluate and improve
cadet leadership performance. Because they are completed by
superior, peer, and subordinate raters, CPR ratings enable a
360° view of leadership performance to be obtained.

Evidence of the content validity of the CPR leadership
dimensions has been provided by prior USMA research. This report
provides evidence of their construct validity for measuring cadet
leadership behavior. This evidence was obtained by examining the
conceptual relationship of CPR dimensions to those found in other
taxonomies of leadership behavior, including those used by the
Center for Army Leadership and the Reserve Officers Training
Corps. Evidence of their construct validity was also obtained by
examining their internal statistical properties and their
statistical relationships with more holistic leadership measures.
Taken as a whole, the evidence suggests the CPR to be a valid
instrument for measuring leadership behavior, with the potential
to generalize to a broad range of leadership situations and
environments.

ZITA M. SIMUTIS EDGAR M. JOHNSON
Deputy Director Director
(Science and Technology)



Table 8

Median Component Loadings of CPR Dimensions Across
19 Principal Components Analyses

Component
Formal Informal Self-
Cognitive Interpersonal Interpersonal Management
Dimension (12) (15) (16) (15)
Duty Motivation 34 -26 +32 81
Military Bearing 34 +25 28 80
Teamwork 46 -38 56 +29
Influencing
Others -56 56 “54 -36
Consideration
for Others 44 +47 +721 17
Professional
Ethics +52 54 +53 -37
Planning and
Organizing +75 .39 20 46
Delegating 66 +73 46 21
Supervising 60 +73 .38 225
Developing
Subordinates 43 64 39 31
Decision Making 273 62 60 +25
Oral and Written
Communication 69 +43 34 .33

Note. Principal components analyses conducted across classes
were based on 1929 cadets for superior raters, 1927 for peers,
1163 for subordinates, and 1317 for instructors. The number of
analyses in which each component emerged is shown in parentheses
and it is, for 9 of the 12 dimensions, identical to the number of
loadings used in calculating each median loading. Because Fourth
Class cadets are not generally rated on three dimensions
(delegating, supervising, and developing subordinates), median
component loadings were based on three fewer loadings for these
dimensions. The median loading of each dimension on its
hypothesized component is underlined.

24



for the purpose of developmental feedback at midterm, only the
final leadership grade, awarded near the end of the term, was
used in our analyses. The final grade represents the official
record of each cadet's overall performance in a leadership role.

The procedure for awarding the leadership grade entails
calculating a weighted average of the individual grades awarded
by a number of raters, usually four. For cadets in most duty
positions, 50% of the final grade is determined by one's Tactical
Officer’, 30% is determined by one's first-level (immediate)
superior in the cadet chain of command, 10% is determined by
one's second-level superior, and the remaining 10% is determined
by one's third-level superior. An important exception to this
general procedure is the fact that Tactical Officers can
determine 100% of the final grade when they believe cadets have
demonstrated marginal (D) or unsatisfactory (F) performance.

Cadet performance report (CPR). The CPR rating instrument
consists of 12 leadership dimensions rated on a 5-point scale
with the following anchor points: Excellent (5), Outstanding
(4), Success (3), Needs Some Improvement (2), Needs Much
Improvement (1). In addition to numerical ratings, at least two
dimensions are selected as comparative strengths ("s") and at
least two as relative weaknesses needing improvement ("n"). An
"x" is awarded to those dimensions considered to be neither a
strength nor a weakness. An overall measure of cadet leadership
performance, referred to as the CPR global score in the remainder
of this report, is also evaluated on a 5-point rating scale. On
this global 5-point scale, cadets are rated as being either in
the Upper 10% (5), Upper 25% (4), Middle 30% (3), Lower 25% (2),
or Lower 10% (1). Raters provide CPR scores on cadet performance
in comparison with all other cadets in a particular duty
position.

The CPR rating procedure at the Academy is automated during
the two academic terms occurring each year (i.e., the Fall and
Spring semesters). Every rater enters their CPR ratings of
others on their own personal computer, which is connected to a
central host computer via a local area network. In this manner,
academic term CPR ratings are centrally collected prior to
subsequent collation and dissemination. In contrast, CPR ratings
are completed manually with a paper-and-pencil instrument during
the Summer term, which is devoted primarily to developing the
military leadership skills of cadets in a field training
environment. The manual version of the CPR rating form is shown
in Appendix A, while an example of a report generated by the
automated system is shown in Appendix B.

2 a Tactical Officer is a military officer charged with
monitoring and shaping individual development within a company of
cadets. .

10



Table 4
i)

Correlations of the Leadership Grade With CPR Dimension Scores
for Superior, Peer, and Subordinate Raters

Type of Rater

Dimension Superior Peer Subordinate
Duty Motivation 37 231 35
Military Bearing .28 -30 30
Teamwork 13 +10 03
Influencing Others 15 12 11
Consideration for Others 06 ~07 O21
Professional Ethics -11 -13 -11
Planning and Organizing 17 20 -08
Delegating 14 08 05
Supervising -13 -121 +03
Developing Subordinates -10 07 11
Decision Making 13 -12 -05
Oral and Written +12 08 03
Communication

Note. Depending on the dimension, correlations were based on
between 2005 and 2375 cadets for superior raters, between 1981
and 2233 for peer raters, and between 1208 and 1275 for
subordinate raters. Correlations of .06 or greater were
statistically significant, p < .01.

16



In summary, it should be noted that peer and subordinate CPR
ratings are measures largely independent of the leadership grade,
because these ratings are made after grades are finalized and
because they involve a largely different set of raters from those
involved in the assignment of leadership grades. However,
leadership grades are somewhat dependent upon the CPR ratings of
immediate superiors, to the extent that each cadet's immediate
superior determines a weighted portion of their final leadership
grade (30% for most duty positions). Further, superior CPR
ratings can be used as one potential source of performance input
by any of the graders (usually four) that determine a particular
cadet's leadership grade. Other potential sources of input to
the leadership grade are discussions about the rated cadet's
performance that are held among these graders, as well as any
personal observations they may have made. Finally, it again
should be noted that the assignment of leadership grades is
governed by a forced distribution system, which limits the number
of high grades awarded. CPR ratings, including superior ratings,
are not governed by a forced distribution system.

Procedure

Leadership grades and automated CPR records were provided to
us for analysis from archival data bases at USMA. These CPR
records were sorted by type of rating into separate files of
superior, peer, subordinate, and instructor ratings. Duplicate
ratings were culled. We then examined each rating to insure it
had been properly classified by type of rating.

To obtain the most current representation of overall term
performance, in relation to the leadership grade assigned near
the end of the term, superior ratings made during the first half
of the term were dropped. For each cadet, in fact, we retained
only the most recent rating provided by their immediate superior
during the second half of the term. Limiting superior ratings to
the second half of the term also resulted in greater temporal
consistency with peer, subordinate, and instructor ratings, which
were all made near the end of the term.

Unlike superior ratings, in which each cadet had only one
assigned rater, the other types of ratings could involve multiple
raters, because each used a nomination process. Thus, one cadet
might receive no peer, subordinate, or instructor ratings at all,
if they were considered to be neither a high nor a low performer.
Conversely, another cadet might receive multiple ratings from
each of these types of raters. For cadets with multiple ratings,
analyses were based on the average scores obtained across raters.

Collectively these procedures yielded the overall cadet
sample shown in Table 2. One will note a relatively lower total
number of cadets rated by subordinates, in comparison with other
types of raters. There were two reasons for the smaller
subordinate sample. First, cadets in the Class of 1997 had no

12



another across all cadet classes and types of raters. Although
our examination of median component loadings tended to minimize
class and rater differences, we now consider how various types of
raters used the dimensions to differentiate cadet performance.

Use of CPR Dimensions by Different Types of Raters

Although no formal hypotheses were tested, we further
investigated the construct validity of the CPR dimensions by
calculating the frequency with which different types of raters
used individual dimensions to describe the relative leadership
strengths and weaknesses of rated cadets. As noted earlier,
raters select at least two dimensions they perceive to be
relative strengths of each ratee and at least two different ones
they perceive to be relative weaknesses. For this analysis,
dimension usage was defined as the percentage of cadets having a
particular dimension rated as a relative strength plus the
percentage having that same dimension rated as a relative
weakness. Table 9 displays the CPR dimensions rank ordered by
dimension usage for different types of CPR raters.

Although dimension percentages can be compared within each
type of rater, they should not be compared across types of raters
due to the inherent procedural differences involved.
Specifically, each cadet received no more than one superior
rating in our investigation. However, each cadet could have
received ratings from many peer, subordinate, and instructor
raters because of the nomination process employed with these
types of ratings. When a cadet received ratings from more than
one rater of a particular type, the dimension scores themselves
were averaged and then rounded to the nearest whole number.
However, the selection of dimensions as relative strengths and
weaknesses was more difficult to classify across multiple raters.
In such cases we chose to classify a dimension as a strength or
weakness only if there was clear agreement among most of the
raters.4 Relative to superior raters, these procedures
artificially lowered levels of dimension usage for peer,
subordinate, and instructor raters. Therefore, when comparing
the dimension usage of different types of raters in Table 9, it
is more appropriate to compare dimension ranks than percentages.

As shown in Table 9, both similarities and differences were
found in the ways the different types of raters used the CPR
dimensions to evaluate the leadership strengths and weaknesses of

4 When a cadet was rated by two or more raters, a dimension
was assigned a value of +1 when it was rated a strength, a value
of -1 when it was rated a weakness, and a value of 0 when it was
rated to be neither a strength nor weakness. Values within each
dimension were then summed across raters. Dimensions were
classified as strengths when this sum was +2 or more and they
were classified as weaknesses when this sum was -2 or less.

25



consideration dimension. Aircraft commanders rated highest in
overall combat effectiveness by their superiors tended to score
above the mean on the two leader behavior dimensions, while those
rated lowest in overall combat effectiveness tended to score
below the mean on both dimensions. Halpin (1953) suggested that
superiors and subordinates selectively perceive "one dimension as
more important than the other, yet in neither case is the second
of the two dimensions viewed adversely" (p. 18). Thus, a
moderately high level of initiating structure was acceptable to
subordinates, as long as their commander was highly considerate.
Conversely, a moderately high level of consideration by
commanders was acceptable to superiors, if it was accompanied by
a high level of initiating structure.

Mount (1984) examined how superior and subordinate ratings
of middle managers on five leadership effectiveness factors
related to the interpersonal satisfaction of subordinates with
their managers. The factors of consideration and expertise were
related to subordinate satisfaction for both superior and
subordinate raters. Their developing employees factor was
related to subordinate satisfaction for subordinates only. Two
factors, administration and structuring work, were unrelated to
subordinate satisfaction for either group. Unfortunately, the
satisfaction of superiors with these managers was not examined.

Cann and Siegfried (1987) asked two groups of undergraduates
to rate an ideal manager of a hypothetical company using a set of
36 traits. Those in one group rated their ideal managers from
the perspective of a superior, while those in the other group
provided ratings from a subordinate perspective. Superiors
tended to rate their ideal managers as more dominant, forceful,
aggressive, analytical, stern, competitive, solemn, and tactful
than did subordinates. However, subordinates tended to rate
their ideal managers as more compassionate, sentimental,
excitable, sensitive, understanding, sincere, and gentle than did
superiors. Both groups of raters described their ideal managers
as being highly reliable, truthful, efficient, and confident.
Later, Cann and Siegfried (1990) suggested superiors value
leaders who are high on initiating structure, while subordinates
value those high on consideration.

Harris and Hogan (1992) found some agreement between
superior and subordinate ratings of managers in a trucking
company using a 55-item questionnaire. In terms of the
relationship of these items to ratings of overall effectiveness,
however, subordinates viewed effective managers as "persons who
can be trusted, who are loyal to the workers, and who work at
building relations with the employees" (p. 17). In contrast,
superiors viewed overall managerial effectiveness largely in
terms of technical competence (cited in Hogan et al., 1994).

In general, these findings indicate the use of different
types of raters in a leadership evaluation program may provide a
more comprehensive view of leadership effectiveness than that



AN EXPLORATION OF THE CONSTRUCT VALIDITY OF A LEADERSHIP BEHAVIOR
RATING SYSTEM

CONTENTS
Page
INTRODUCTION ........5-6- pec er ss ees nce e ewer eee eneee secre eeeee 1
Leadership Dimensions in the U.S. Army .......eeeeeeee eens 2
Effects of Rater Position on the Evaluation Process.......5
Objectives ........ ee rrr) eee eeee 8
METHOD .... ee eee eee eee eeee
fk-011) ok -
Measures ..
Procedure
RESULTS 2. cee cece ec ee eee tee eee eee eee nent seen e ee eenees 14
Relationships Between CPR Dimension Scores
and Holistic Measures of Leadership Performance ........ 14
Interrelationships Among the CPR Dimensions ............-- 20
Use of CPR Dimensions by Different Types of Raters..... ~. 25
DISCUSSION .. cc eee ec cet eee ee eee ete ee nee eee tence eee eee nee 28
REFERENCES 2... cc eee eee ee ee eee eee etree reer erect eee ee eeeeee 35
APPENDIX A. CADET PERFORMANCE REPORT (CPR) RATING FORM...... A-1

B. AN EXAMPLE (FICTIONAL) OF A COMPLETED
CPR FROM THE AUTOMATED SYSTEM .........-- see ewes B-1
LIST OF TABLES

Table 1. A Comparison of Leadership Dimensions From
Three U.S. Army Classification Systems ............ 4

2. Number of Cadets Rated by Class and
Type of CPR Rater...... see eeeeee wee eee eer e eens 13

3. Means and Standard Deviations of CPR Dimension

Scores and the CPR Global Score for Superior,
Peer, and Subordinate Raters ......ee eee cece eee 15

xi



hypothesize the presence of four major components underlying the
dimensions. We considered a dimension to be part of a particular
component if it had a higher relative loading on that component
than on any of the other components (.20 or greater difference).

The first component appeared related to cognition, because
it contained the dimensions of planning and organizing, decision
making, and oral and written communication. Interpersonal in
nature, the second component seemed to be characterized by those
role-related activities that formal or designated leaders of a
group are usually expected to perform. This formal interpersonal
component consisted of the dimensions of delegating, supervising,
and developing subordinates. Also interpersonal in nature, the
third component appeared related to leadership activities of a
more informal nature, in the sense that they could be performed
by any group member, not just by formal leaders. This informal
interpersonal component consisted of the dimensions of teamwork,
influencing others, consideration for others, and professional
ethics. Finally, the fourth component appeared related to the
concept of self-management. Least interpersonal in its
orientation, this self-management component contained the
dimensions of duty motivation and military bearing. Although not
evident solely from the empirical results of the principal
components analyses, our subjective appraisal of numerous rater
comments suggests these latter two dimensions are related to the
notions of task accomplishment, effort (toward both individual
and organizational goals), perseverance, and the maintenance of a
professionally appropriate demeanor.

The entire set of dimensions associated with an underlying
component did not surface in every analysis. Therefore, in some
analyses a component may not have been as completely represented
as in other analyses. For example, only two of the three
dimensions comprising the formal interpersonal component (e.g.,
delegating and supervising) may have loaded together on the same
component in a particular analysis. In cases such as these, we
considered a component to be present if at least two of its
hypothesized dimensions loaded on the same component together.

To investigate whether these four components adequately
describe the underlying conceptual structure of CPR ratings,
median component loadings across the 19 analyses were calculated
for each of the 12 CPR dimensions. To better -illustrate the
procedures used in our calculation of median component loadings,
consider as an example the overall analysis of instructor ratings
across classes, in which a two-component structure emerged.

Three dimensions in this analysis (delegating, supervising, and
developing subordinates) had relatively higher loadings on the
first component than on the second component (.20 or greater
difference). Therefore, the loadings of the 12 dimensions on the
first component were used as one set of loadings in calculating
the median loadings for the formal interpersonal component.
Conversely, five dimensions (duty motivation, military bearing,
planning and organizing, decision making, and oral and written

22



Table 6

Correlations of the CPR Global Score With CPR Dimension Scores

for Superior, Peer, and Subordinate Raters

Type of Rater

Dimension Superior Peer Subordinate
Duty Motivation 53 -65 56
Military Bearing ~43 51 49
Teamwork 25 46 235
Influencing Others +23 37 46
Consideration for Others 15 37 -34
Professional Ethics 16 231 +30
Planning and Organizing +25 30 23
Delegating 16 -18 +21
Supervising 19 22 23
Developing Subordinates +21 30 46
Decision Making +221 +24 19
Oral and Written +22 23 +22

Communication

Note. Depending on the dimension,

correlations were based on

between 2004 and 2375 cadets for superior raters, between 1978

and 2238 for peer raters,

and between 1219 and 1285 for

subordinate raters. All correlations were statistically

significant, p < .001.

19



AN EXPLORATION OF THE CONSTRUCT VALIDITY OF A LEADERSHIP BEHAVIOR
RATING SYSTEM

EXECUTIVE SUMMARY

Research Requirement:

An effective method of measuring leadership behavior was
needed as an essential part of a longitudinal program of
leadership development research recently initiated by the Center
for Leadership and Organizations Research (CLOR) at the U.S.
Military Academy (USMA). One measure being considered for its
ability to fulfill this key research requirement is the Cadet
Performance Report (CPR), a 12-dimension leadership behavior
rating system currently used to develop and evaluate the
leadership performance of USMA cadets. Although evidence for the
content validity of the CPR dimensions has been provided by prior
USMA research, no previous investigation has examined their
construct validity for measuring cadet leadership behavior.

Procedure:

Records of CPR ratings, each consisting of 12 individual
dimension scores and a global score of leadership performance,
were received from an automated data base at USMA. Separate
analyses of these records, reflecting the CPR ratings of over
2,000 different cadets, were conducted for three different types
of CPR raters: superiors, peers, and subordinates. Our general
approach in exploring the construct validity of the CPR
dimensions was primarily inductive in nature and it involved
three interrelated steps. First, we examined how the CPR
dimensions related to more holistic measures of leadership
performance, specifically the CPR global score and the leadership
grade. The leadership grade is USMA's official evaluation of a
cadet's overall performance in a leadership role. Second, we
investigated the interrelationships existing among the 12 CPR
dimensions themselves in an effort to better understand the
underlying conceptual structure of the CPR instrument. Finally,
we explored how the three types of CPR raters used the dimensions
to differentially evaluate the leadership strengths and
weaknesses of rated cadets.

Findings:

For each type of rater, significant relationships were found
between most CPR dimensions and both the leadership grade and the

vii



ACKNOWLEDGMENTS

We would like to express our gratitude to LTC Robert J.
Lamb, Chief, and Ms. Judy Oswald, Evaluation Officer, of the
Leader Development Branch at the U.S. Military Academy. They
have assisted us in many ways, to include their collecting and
providing us with all Cadet Performance Report ratings and
leadership grades examined in this report. Their continuing
support of our research program remains invaluable and is very
much appreciated.



stemming from the perspective of superiors alone. Although some
similarity is often found, different types of raters appear to
value different aspects of leadership performance. Knowledge of
the organizational position of the rater, relative to the ratee,
may suggest those dimensions likely to be most salient for
particular types of raters in a leadership evaluation program.

Objectives

The overriding goal of the present research was to determine
if the twelve CPR dimensions of USMA's leadership development
rating system have enough construct validity to be used in a
recently initiated longitudinal program of leadership development
research at the Center for Leadership and Organizations Research.
Although their level of content validity has been established by
previous USMA research (OIR, 1989), these dimensions have not
been examined for their level of construct validity in evaluating
leadership performance.

Campbell (1976) maintains that there are both deductive and
inductive approaches for examining the construct validity of a
measure and that the appropriateness of an approach depends upon
the research objectives. Rather than positing formal hypotheses
based on theory, an inductive approach is more appropriate when
one is concerned with giving substantive meaning to a particular
set of measures for which one is not yet ready to predict
empirical relationships (Campbell, 1976). We selected an
inductive approach to examine the construct validity of the CPR
dimensions based upon these guidelines and the exploratory nature
of our work.

Our objectives in exploring the construct validity of the CPR
dimensions were threefold. First, we wanted to determine the
relationships, if any, of these dimensions to more holistic
measures of leadership performance. Second, we wanted to
describe the nature of potential interrelationships existing
among the twelve dimensions, in order to better understand the
CPR's underlying conceptual structure. Finally, we wanted to
determine if different types of raters diverge in how they use
the dimensions to evaluate leadership strengths and weaknesses.



leadership development over time. To the extent that these
dimensions overlap with those in other leadership taxonomies, our
findings should have some general applicability to a broader
range of leadership situations and environments.

ix



Results

Table 3 presents the means and standard deviations of CPR
dimension scores and the CPR global score for superior, peer, and
subordinate raters. Overall, the means and standard deviations
were similar across the three types of raters. As expected, the
mean scores of peers and subordinates were slightly lower than
those of superiors due to CPR completion requirements for those
raters (i.e., more nominations of cadets with relatively low
levels of performance than cadets with relatively high levels of
performance). Among the 12 CPR dimensions, duty motivation and
military bearing received the highest mean scores and standard
deviations for each type of rater. Most of the dimension means
fell between Success (3) and Outstanding (4) on a 5-point scale.
However, the global score mean for each type of rater tended to
be closer to Upper 25% (4) on a 5-point scale. Similarly, the
leadership grade was found to have a mean of 2.95 (almost a B)
and a standard deviation of .62. Like the CPR global score, the
leadership grade represents a holistic assessment of performance.
Unlike the CPR global score, the leadership grade was assigned
using a forced distribution system that limited the number of
high grades given (as mentioned earlier).

Relationships Between CPR Dimension Scores and Holistic Measures
of Leadership Performance

Leadership grade. Table 4 summarizes the relationships
found between the leadership grade and CPR dimension scores for
superior, peer, and subordinate raters. It is apparent when
comparing across types of raters that many of these relationships
were slightly stronger for superiors and peers than for
subordinates. In particular, the dimensions of teamwork,
consideration for others, delegating, supervising, decision
making, and oral and written communication were significantly
related to the leadership grade only for superior and peer raters
(p < .01). The lack of significant relationships for
subordinates on these dimensions suggests their conception of
leadership behavior was somewhat different from that of superiors
and peers.

In addition, the relationship found between the leadership
grade and the CPR global score, across types of raters, also
suggested somewhat distinct rater perspectives. These variables
correlated more strongly for superiors (r=.57) than for peers
(r=.41) and subordinates (r=.35). The presence of a stronger
relationship for superior raters was anticipated because of the
senior position held both by cadets providing the CPR rating and
those raters supplying the leadership grade (cadets and Tactical
Officers). Despite differences in magnitude for each type of
rater, all correlations were statistically significant (p <
001), indicating some similarity between these two holistic
measures of leadership performance.

14



aa

Technical Report 1041

| An Exploration of the Construct Validity of a
Leadership Behavior Rating System

Elisa H. Schwager and Kenneth L. Evans
U.S. Army Research Institute

April 1996
United States Army Researcn Institute
for the Behavioral and Social Sciences
Lesh are WSU

Approved for public release; distribution is unlimited.

DTIC Guasave Morac



Appendix A

Cadet Performance Report (CPR) Rating Form



AN EXPLORATION OF THE CONSTRUCT VALIDITY
OF A LEADERSHIP BEHAVIOR RATING SYSTEM

Introduction

Identifying the behaviors that represent the leadership
domain and determining their relevance to successful performance
are essential for the advancement of leadership theory and the
improvement of leader development programs. Leadership behavior
rating systems provide a framework for defining and measuring
various dimensions of leadership effectiveness, thereby
facilitating these goals. In examining the construct validity of
any leadership behavior rating system two primary concerns must
be addressed. First, the respective meaning of the dimensions
being measured must be considered, both as they relate to one
another and as they relate to external criteria, often of a more
holistic nature. Second, the effects on the evaluation process
of the rater's organizational position, relative to the ratee,
also must be taken into account.

Historically, leadership researchers have explored the
meaning and interrelatedness of leadership dimensions through the
creation of taxonomies that attempt to categorize behaviors for
effective leadership across situations. Fleishman et al. (1991)
reviewed 65 different classifications of leadership behavior
found between 1940 and 1986. Taxonomies and classification
systems such as these have provided both a means for organizing
leadership terms and concepts, as well as a mechanism for
communication (Clement & Ayres, 1976).

Although taxonomies differ in terms of the number and types
of dimensions posited, some consistent trends are found across
many of the classification systems. For example, two dimensions
found in almost every taxonomy are facilitating group relations
and accomplishing tasks (Fleishman et al., 1991). Showing
concern for subordinates and maintaining standards of performance
are exemplary of behaviors fitting into these classifications.
Although the dimensions associated with such behaviors are
probably best known as consideration and initiation of structure,
from the Ohio State University leadership studies of the early
1950s (see Fleishman, 1973), they are listed in a variety of
leadership taxonomies under different labels.~

In addition, the managerial and administrative functions of
leadership appear quite uniformly across taxonomies. As noted by
Fleishman et al. (1991), these functions involve monitoring both
personnel and physical resources. Kanungo and Misra (1992) have
also noted that behaviors involved in resource planning and
coordination are evident in many leadership frameworks. Yet,
Kanungo and Misra (1992) distinctly emphasize the importance of
managing oneself. In their conceptualization, self-management
has a central role in regulating the effectiveness of a leader's
performance in all other areas.



CONTENTS

(Continued)

Table

Page
LIST OF TABLES (Continued)

Correlations of the Leadership Grade With
CPR Dimension Scores for Superior, Peer,
and Subordinate Raters ... eee eee eee eee eee eee 16

Summary of Stepwise Multiple Regression

Analyses Predicting the Leadership Grade

From CPR Dimension Scores for Superior,

Peer, and Subordinate Raters ..... eee ee eee eee eee 18

Correlations of the CPR Global Score With
CPR Dimension Scores for Superior, Peer,
and Subordinate Raters 2... ee eee eee eee ee eee eee 19

Summary of Stepwise Multiple Regression

Analyses Predicting the CPR Global Score

From CPR Dimension Scores for Superior,

Peer, and Subordinate Raters ..... eee ee eee eee eee 21

Median Component Loadings of CPR
Dimensions Across 19 Principal
Components Analyses....... eee ee ee eee eee eee . 24

Rank Order of CPR Dimension Use by

Superior, Peer, Subordinate,
and Instructor Raters .............-- see ee eee e eee 26

xii



those factors that most influence the process by which cadets are
selected for duty positions in the last two years before their
graduation.

In summary, our exploratory findings suggest the CPR
dimensions have a sufficient level of construct validity for the
purpose of measuring cadet leadership behavior over time. The
support for this view comes from three areas of investigation.
First, the dimensions are related to, though they are not the
same as, more holistic measures of leadership performance.
Second, the dimensions interrelate among themselves in ways that
are explainable, meaningful as they relate to other
classifications of leadership behavior, and generally consistent
across a broad array of separate analyses. Third, the divergent
ways that various types of raters use the dimensions to evaluate
leadership performance are much like those that have been
documented in the literature.

We believe these results are encouraging. Therefore, we
plan to use the CPR as a central measure of leadership behavior
in a recently initiated longitudinal program of leadership
development research at USMA. CPR ratings appear to bea
complementary adjunct to the leadership grade because they
provide additional information about the nature of leadership
performance, particularly in those areas not typically stressed
by superiors. As they provide more detailed and comprehensive
information than holistic measures alone, CPR ratings appear
well-suited to the task of evaluating cadet leadership
development over time. To the extent the 12 CPR dimensions
overlap with those found in other leadership taxonomies, our
findings should have some general applicability to a broader
range of leadership situations and environments.

33



Table 7

Summary _of Stepwise Multiple Regression Analyses Predicting the

CPR Global Score From CPR Dimension Scores for Superior, Peer,
and_ Subordinate Raters

Dimension AR? R?
Superior
Duty Motivation +26 +26
Military Bearing 05 231
Planning and Organizing 02 33
Teamwork 201 234
Peer
Duty Motivation 42 «42
Military Bearing 06 48
Teamwork 05 53
Consideration for Others O01 54
Influencing Others -O1 55
Planning and Organizing -O1 ~56
Subordinate
Duty Motivation 231 31
Influencing Others 10 41
Developing Subordinates .05 46
Military Bearing 03 249
Teamwork 02 51
Consideration for Others OL 52

Note. Regression analyses were based on 1923 cadets for
superior raters, 1927 for peer raters, and 1155 for subordinate
raters. The incremental variance explained by each of the above
dimensions was statistically significant, p < .0001.

21



CADET PERFORMANCE REPORT
For Use of this Form See USCC Regulation 623-1; Proponent Agency is LDB, USCC.

PART I - ADMINISTRATIVE DATA

a. NAME (Last, First, Middle Initia) b, SSAN < GRAD YR &_AY COMPANY
© RANK — DUTY ASSIGNMENT ORGANIZATION
h, TYPEREPORT: Q student peer 0 subordinate O) chain of command Q other

PART II - AUTHENTICATION
3 NAME OF RATER (Last First Middle Initial) b, SIGNATURE ©. DATE
7 RANK © DUTY ASSIGNMENT — ORGANIZATION
@ NAME OF OFFICER REVIEWER (Last First Middle inital) h SIGNATURE 7, DATE
7 RANK Kk DUTY ASSIGNMENT T. ORGANIZATION
im RATED CADET: [have seen this report completed through Part IV. I verify | n. SIGNATURE 0. DATE
entries in Part I. understand my signature does not constitute agreement with the
evaluation of the rater, Tdo donot wish to make a statement in my own
bschalf concerning the rater's evaluation (attach statements).

PART III - PERFORMANCE OF DUTIES (Rater)

Rate each of the following dimensions using the following scale: 5 Excellent, 4 - Outstanding, 3 - Successful, 2-Needs Some Improvement, I - Needs
Much Improvement. Specific Bullet examples are required for ratings other than "Success." Comments on Success ratings are optional, Indicate the
dimensions (at least two) which are comparatively strongest by checking the "S" block and those (at least two) requiring relatively more attention by checking
the "N" block. Comments are required for those dimensions marked "S" or "N.

50
40
30
20
10

a. DUTY MOTIVATION: Actions that indicate persistence in the attempt to achieve high standards of performance for self, subordinates,
and others

5a
40
30
20
1a

b. MILITARY BEARING: Maintaining Army standards of appearance, physical fitness, manner, composure, and courtesy.

50
4a
3a
20
1a

c. TEAMWORK: actions that indicate commitment to the achievement of organizational goals while working effectively with others; support of
organizational rules and regulations.

5a
40
30
20
10

d. [INFLUENCING OTHERS: The act of using appropriate interpersonal styles and methods in guiding individuals or groups towards task

accomplishment or resolution of vonfliets and disagreements.

5a
4a
30
20
10

CONSIDERATION FOR OTHERS: actions which indicate a sensitivity to and regard for the feelings and needs of others and an

jor on them, deing supportive of and fair with others,

oY%Jo2z ovjo4 ovo2 o702 OF

wvareness af the effect on one’s own beha

50
4a
30
20
10

€ PROFESSIONAL ETHICS: stantaining ethical, moral, and Army protessional standards and vaiues. accepting and acknowledging full

responsibility for one’s actions and their consequences,

o4z o%/02

USMA Form 2-543R (Oct 93)



statistically for superiors [F(6,1916)=168.74, p < .0001], peers
(FE (9,1917)= 269.87, p < .0001], and subordinates [F(8,1146)=
157.82, p < .0001]. Each of these models explained between 34%
and 56% of the variation in CPR global scores. As shown in Table
7, scores on the duty motivation dimension explained most of this
variation for each type of rater.

Nevertheless, differences among the types of raters were
found in the relative contributions of other CPR dimensions to
CPR global score variation. Except for duty motivation, military
bearing explained more additional global score variation for both
superior and peer raters. For subordinate raters, however,
influencing others and developing subordinates explained more
additional variation than did military bearing. Teamwork
contributed to the model for each type of rater, especially
peers. The planning and organizing dimension also contributed to
the models for superiors and peers, as did the consideration for
others dimension for peers and subordinates. In addition, the
influencing others dimension contributed to the model for peers,
though to a lesser extent than it did for subordinates. Other
dimensions comprising the models explained a relatively small
amount of additional variation (a total of less than 1% in each
model). As a whole, the dimensional differences among types of
raters were more pronounced for CPR global score prediction than
they were for leadership grade prediction (see Table 5).

Interrelationships Among the CPR Dimensions -

A series of 19 principal components analyses with promax
rotation was employed to determine if the 12 CPR dimensions could
be characterized as having broader underlying factors. For each
of four types of raters (superiors, peers, subordinates, and
instructors), separate analyses were conducted by cadet class,
with the exception that there were no subordinate ratings of the
Fourth Class to analyze. In addition, an overall analysis across
classes was conducted for each type of rater. These 19 analyses
produced a variety of component structures: 9 two-factor
structures (M% variance explained=42%,12%), 8 three-factor
structures (M% variance explained=30%,11%,9%), and 2 four-factor
structures (M% variance explained=34%,12%,10%,9%). However, a
synthesis of the results from the separate analyses led us to

3 Analyses conducted within cadet class for each type of
rater also demonstrated the greater relative importance of duty
motivation in explaining CPR global score variation. One
exception to this general trend was found. In Fourth Class
subordinate ratings of Third Class cadets, developing
subordinates explained more global score variation (31%) than any
other dimension. As the primary leadership task of Third Class
cadets is to develop a subordinate cadet in the Fourth Class, the
extent to which CPR ratings reflect this leadership emphasis is a
positive indication of their construct validity.

20



CPR global score. In particular, the dimensions of duty
motivation and military bearing were most strongly related to
these more holistic measures of leadership performance across
rater types. Relative to the other 10 CPR dimensions, they also
explained considerably more of the variation in the holistic
measures, though there was some deviation from this general trend
for subordinate raters.

A series of principal components factor analyses suggested
the 12 CPR dimensions have an underlying structure consisting of
four broader components of leadership behavior. Specifically,
there appears to be a cognitive component related to three CPR
dimensions (planning and organizing, decision making, and oral
and written communication), a formal interpersonal component
related to three dimensions (delegating, supervising, and
developing subordinates), an informal interpersonal component
related to four dimensions (teamwork, influencing others,
consideration for others, and professional ethics), and a self-
management component related to two dimensions (duty motivation
and military bearing).

Both similarities and differences were found in the ways the
three types of raters used the CPR dimensions to evaluate the
leadership strengths and weaknesses of rated cadets. Duty
motivation and military bearing were the most often used CPR
dimensions among the three types of raters. However, differences
were found along most of the other 10 dimensions. For example,
superiors were more likely than either peers or subordinates to
use the dimension of supervising. In addition, superiors were
more likely than subordinates to use the dimension of planning
and organizing, as well as the dimension of oral and written
communication. In contrast, both peers and subordinates were
more likely than superiors to use the dimension of consideration
for others. Further, subordinates were more likely than either
superiors or peers to use the dimensions of influencing others
and developing subordinates.

Utilization of Findings:

Our exploratory findings have substantial consistency with
related research in the literature and provide support for the
construct validity of the CPR dimensions as measures of
leadership behavior. Their usefulness in obtaining multiple
views of leadership performance was demonstrated, particularly in
those areas not typically stressed by superiors. Because they
provide more detailed and comprehensive information about cadet
leadership behavior than holistic measures alone, the CPR
dimensions appear well suited to the task of evaluating cadet

viii



Cadets are the major provider of CPR ratings at USMA. They
evaluate other cadets from superior, peer, and subordinate rating
positions. Superior ratings are provided by cadet leaders who
are assigned to rate those in subordinate duty positions. These
rating assignments are based upon the duty position and class of
the cadet leader. Although a cadet may receive CPR ratings from
superiors at several different organizational levels, our
investigation focused on those ratings provided by first-level
(immediate) superiors. Immediate superiors are expected to rate
their assigned subordinates at least once during each half of the
academic term, though our analyses considered only the last
rating given to each cadet during the second half of the term.
For most cadets, this rating occurred just prior to the award of
the final leadership grade.

In contrast, peer and subordinate CPR ratings are based on a
nomination process. Rather than having assigned cadets to rate,
peer and subordinate raters must decide whom to rate themselves,
within certain procedural requirements. In general, they are
instructed to nominate and rate those having the highest and
lowest relative levels of performance within a particular group
of cadets. Yet, the actual numbers of cadets rated vary
according to the type of rating and the class of the rater.
Specifically, peers in the First Class (seniors) and the Second
Class (juniors) are instructed to rate the highest performer and
the two lowest performers within their company and respective
class. Peers in the Third Class (sophomores) and Fourth Class
(freshmen) rate the highest performer and the two lowest
performers within their platoon and respective class. In
comparison, subordinates in the Third Class rate the two highest
and three lowest Second Class cadets in their company, while
subordinates in the Fourth Class rate the highest and two lowest
Third Class cadets in their platoon. Although not required,
Second Class cadets have the option of giving subordinate CPR
ratings to First Class cadets. Despite these procedural
differences, all peer and subordinate ratings are made near the
end of the academic term, following the award of final leadership
grades. The identity of peer and subordinate raters is unknown
to rated cadets, who later receive developmental counseling by
their Tactical Officer on the contents of the peer and
subordinate ratings.

Additional CPR ratings are given by non-cadet raters,
including academic instructors, coaches, and club sponsors.
Rating procedures for instructors are similar to those for peers
and subordinates. Near the end of each academic term instructors
are asked to nominate and rate the highest and lowest relative
performers among the cadets enrolled in their academic courses.
We examined instructor ratings for their utility in providing
supplementary information about the construct validity of the CPR
dimensions. However, CPR ratings provided by coaches and club
sponsors were not examined because of the comparatively small
number of ratings involved.

11



Discussion

Each of the 12 CPR dimensions was positively related to the
holistic measures of leadership performance in our investigation,
though some dimensions were more strongly related than others.
Specifically, the dimensions of duty motivation and military
bearing had a central role in explaining most of the variation
found in both the leadership grade and the CPR global score.

This is consistent with Kanungo and Misra's (1992) view that
effective self-management is critical to leadership performance
in all other areas. Morgan (1989) also found that motivation
ratings explained most of the variation in a holistic measure of
leadership performance.

It is possible that cadet raters viewed duty motivation and
military bearing as more important contributors to effective
cadet leadership performance or that they were just more easily
observed than many of the other 10 dimensions. These inferences
are partially supported by the findings of a content validity
study in which subject matter experts (cadets, tactical officers,
faculty, and staff) rated how essential and observable the 12 CPR
dimensions were in evaluating cadet performance in a variety of
leadership roles (OIR, 1989). Averaging across all duty
positions, they found duty motivation to be the dimension rated
most essential to successful cadet performance. They also found
military bearing to be the most observable dimension, followed by
the oral and written communication dimension and the duty
motivation dimension.

It appears the duty motivation and military bearing
dimensions are highly applicable to evaluations of almost any
cadet leadership task. Conversely, the other 10 dimensions seem
more specific in their orientation, with less general
applicability across situations. Some obvious examples of the
latter dimensions are supervising, delegating, and developing
subordinates. As mentioned earlier, these three dimensions were
rarely used by any type of rater to evaluate cadets in the Fourth
Class. Further, they were not used very often by instructors to
evaluate cadets in any class. Relative to other CPR dimensions,
duty motivation and military bearing are probably more analogous
to the holistic measures of leadership performance because of
their greater level of general applicability across situations.

Even though duty motivation and military bearing have a
clear role to play in cadet leadership evaluation, their
importance was not universally apparent. In the principal
components analyses, for example, the self-management component
tended to account for a generally smaller percentage of the
dimensional variance than the other three components. Consisting
of duty motivation and military bearing, the self-management
component was the first component to emerge in only 3 of the 15
analyses in which it appeared. In addition, the dimensions of
influencing others and developing subordinates were more
important than military bearing in explaining CPR global score

28



variation among subordinate raters. Among Fourth Class
subordinate raters of Third Class cadets, developing subordinates
explained more global score variation than either duty motivation
or military bearing.

The finding that CPR dimensions were all positively related
to more holistic leadership measures suggests the dimensions did,
in fact, measure some aspect of leadership performance. Yet
consistently, the dimensions were more strongly related to the
CPR global score than to the leadership grade. There are several
reasons why the dimensions were more closely related to the
global score. First, a cadet's CPR global score was provided by
the same person providing that cadet's CPR dimension scores. In
contrast, a cadet's leadership grade was based on the weighted
judgments of a group of superiors, only one of which (i.e., the
immediate superior) also provided CPR dimension scores of that
cadet's performance. Second, a cadet's global score was provided
at the same point in time as the accompanying dimension scores,
whereas that cadet's leadership grade was provided at a slightly
different point in time. Finally, a cadet's leadership grade
could have been affected by the forced distribution system of
grading, unlike either the CPR global score or the dimension
scores.

Despite the inherent procedural differences between these
two holistic measures, we found them to be significantly related
to each other, particularly for superior raters. It is even
possible to consider the global score from a superior CPR rating
to be conceptually equivalent to a leadership grade that might be
assigned by an immediate superior without a forced distribution
system. In this hypothetical example, both holistic measures
would be provided by the same person using similar five-point
rating scales. There are few reasons to expect these two
measures might yield widely differing results.

The CPR dimensions themselves were interrelated to some
degree, as reflected by the emergence of a four-component factor
structure. We conducted the series of 19 principal components
analyses, not for the purpose of data reduction, but to better
understand how the dimensions relate to one another conceptually.
Though our calculation of median component loadings tended to
minimize the effects of class and rater differences, the four-
component structure appears to provide some insight into how
raters view the cadet performance domain and it appears to fit
within broader conceptual frameworks of leadership advanced in
the literature (Fleishman, 1973; Fleishman et al., 1991; Korotkin
et al., 1986).

In particular, the view of cadet performance obtained from
the regression analyses was entirely different from that obtained
from the principal components analyses. Focusing on the
relationships of CPR dimensions to other measures, the regression
analyses seemed to initially suggest that many raters saw cadet
leadership performance as little more than duty motivation and

29



In order to investigate the role of CPR dimensions in
explaining leadership grade variation, a stepwise multiple
regression analysis was conducted for each type of rater. The
resulting multidimensional models were statistically significant
for superiors [F(6,1913)=60.69, p < .0001], peers (E(5,1923)=
70.92, p < .0001], and subordinates [F(4,1145)=53.38, p < -0001).
Each of these models explained between 15% and 16% of the
variation in leadership grades. As shown in Table 5, most of
this variation was explained by the dimensions of duty motivation
and military bearing for each type of rater. The planning and
organizing dimension contributed to the models for superiors and
peers, as did the consideration for others dimension for
subordinates. Other dimensions comprising the models explained a
relatively small amount of additional variation (a total of less
than 1% in each model).

CPR global score. Table 6 shows the relationships found
between the CPR global score and the CPR dimension scores for
superior, peer, and subordinate raters. For each type of rater,
the CPR global score was most strongly related to the duty
motivation and military bearing dimensions. However, the
strength of the relationship between the global score and many of
the other dimensions varied across the three types of raters.

For example, teamwork, influencing others, consideration for
others, professional ethics, and developing subordinates were all
more strongly related to the global score for peers and
subordinates than for superiors. These findings suggest peers
and subordinates attributed greater relative importance to such
behaviors in their evaluations of overall leadership performance.
They may also have had more opportunities than superiors to
observe such behaviors.

A comparison with Table 4 reveals that CPR dimension scores
were more strongly related to the CPR global score than to the
leadership grade. One reason for this finding is that for every
cadet rated, dimension scores and the global score are given by
the same rater at the same point in time. Conversely, the
leadership grade is based upon judgments from several raters,
each having partial input to the assigned grade. In addition,
the individual contribution to the leadership grade made by a CPR
superior rater (usually about 30%) is not necessarily reflected
in the final grade after averaging across all graders (peers and
subordinates have no direct input to the leadership grade). As
noted earlier, the leadership grade is assigned according to a
forced distribution system that restricts the amount of A's, B's,
and C's awarded to cadets. Consequently, the strength of the
relationships found between the dimension scores and the
leadership grade may also be restricted.

In order to investigate the role of CPR dimensions in
explaining CPR global score variation, a stepwise multiple
regression analysis was conducted for each type of rater. The
resulting multidimensional models were highly significant

17



Technical Report 1041

An Exploration of the Construct Validity of a Leadership
Behavior Rating System

Elisa H. Schwager and Kenneth L. Evans
U.S. Army Research Institute

Leader Development Research Unit
Trueman R. Tremble, Jr., Chief

U.S. Army Research Institute for the Behavioral and Social Sciences
5001 Eisenhower Avenue, Alexandria, Virginia 22333-5600

Office, Deputy Chief of Staff for Personnel
Department of the Army

April 1996

Army Project Number Education and Training Technology
20262785A791

Approved for public release; distribution is unlimited.



Twohig, P.T., & Tremble, T.R., Jr. (1991). Leadership performance
measurement _in a tactical environment (ARI Research Report
1580). Alexandria, VA: U.S. Army Research Institute for the
Behavioral and Social Sciences. (AD A232 792)

Tremble, T.R. (1992). Relationships of leadership competence with
leader and unit performance effectiveness. (ARI Research
Report 1625). Alexandria, VA: U.S. Army Research Institute
for the Behavioral and Social Sciences. (AD A257 240)

U.S. Army Cadet Command (1993). Cadet command assessor's
handbook. Fort Monroe, VA: U.S. Army Training and Doctrine
Command.

U.S. Corps of Cadets (1995). Leadership evaluation and
developmental ratings (USCC Regulation 623-1). West Point,
NY: U.S. Military Academy.



References

Campbell, J.P. (1976). Psychometric theory. In M.D. Dunnette
(Ed.), Handbook of industrial and organizational psychology
(pp. 185-222). Chicago: Rand McNally.

Cann, A., & Siegfried, W.D. (1987). Sex stereotypes and the
leadership role. Sex Roles, 17, 401-408.

Cann, A., & Siegfried, W.D. (1990). Gender stereotypes and
dimensions of effective leader behavior. Sex Roles, 23, 413-
419.

Clement, S.D., & Ayers, D.B. (1976). A matrix of organizational
leadership dimensions (Leadership Monograph Series No. 8).
Fort Monroe, VA: U.S. Army Administration Center.

Department of the Army (1990). Military leadership (Field Manual
22-100). Washington, DC: Author.

Fleishman, E.A. (1973). Twenty years of consideration and
structure. In E.A. Fleishman & J.G. Hunt (Eds.), Current
developments in the study of leadership (pp. 1-37).
Carbondale, IL: Southern Illinois University Press.

Fleishman, E.A., Mumford, M.D., Zaccaro, S.J., Levin, K.Y.,
Korotkin, A.L., & Hein, M.B. (1991). Taxonomic efforts in
the description of leader behavior: A synthesis and
functional interpretation. Leadership Quarterly, 2, 245-287.

Halpin, A.W. (1953). Studies in aircrew composition: Study 3. The
combat leader behavior of B-29 aircraft commanders (HFORL
Memorandum TN-54-7). Washington, DC: Human Factors
Operations Research Laboratories (AD 54071)

Harris, G., & Hogan, J. (1992, April). Perceptions and

personalities of effective managers. Paper presented at the
13th Annual Psychology in the Department of Defense

Symposium, Colorado Springs, CO.

Harris, M.M., & Schaubroeck, J. (1988). A meta-analysis of self-
supervisor, self-peer, and peer-supervisor ratings.

Personnel Psychology, 41, 43-62.

Hogan, R., Curphy, G., & Hogan, J. (1994). What we know about
leadership: Effectiveness and personality. American
Psychologist, 49, 493-504.

Hollander, E.P. (1957). The reliability of peer nominations under
various conditions of administration. Journal of Applied
Psychology, 41, 85-90.

35



Table 1

A Comparison of Leadership Dimensions From Three

U.S.

Army Classification Systems

CAL
Department of the Army (1990)

LAP
U.S. Army Cadet Command (1993)

CPR
U.S. Corps of Cadets (1995)

Communications Oral Commnication Oral and Written Communication
Written Communication
Oral Presentation

Supervision Administrative Control Supervising
Delegation Delegating

Teaching & Counseling

see Influence

Developing subordinates

Soldier & Team Development

Influence*
Sensitivity
Followership

Influencing Others
Consideration for Others
Teamwork

Technical & Tactical
Proficiency

Technical/Tactical Competence

Decision Making

Problem Analysis
Judgment

Decisiveness

Decision Making

Planning

Planning and Organizing

Planning & Organizing

Use of Available systems”

Professional Ethics

see Influence

Professional Ethics

Initiative
Physical Stamina®
Mission Accomplishment

Duty Motivation
Military Bearing

a

and coaching subordinates.

to being honest and ethical.

Part of the LAP's influence dimension relates to teaching
Part of this dimension also relates

Competency regarding information management technology is
a primary feature of this CAL dimension.

c

This LAP dimension includes the ability to complete

required tasks while under physical, mental, and emotional stress

(i.e., maintaining composure).

In addition,

part of the

influence dimension relates to displaying self-confidence in

one's posture,

appearance,

and behavior.



the simultaneous use of differing types of raters to evaluate the
performance of leaders has gained considerable acceptance in
recent years, at least in theory if not in practice. Hogan,
Curphy, and Hogan (1994), for example, believe the most practical
way to evaluate leaders is to ask their superiors, peers, and
subordinates to rate their performance. The leadership behavior
rating system at USMA, based on the twelve CPR dimensions, is an
example of an evaluation program in which the same dimensional
scheme is used by different types of raters (i.e., superiors,
peers, and subordinates) to assess a cadet's leadership
performance.

Evaluation programs having multiple types of raters can
provide reliable, credible, comprehensive, and informative
feedback to developing leaders that they can then use to organize
self-perceptions about their own leadership performance over time
(Morgan, 1989). In their meta-analytic review of multiple rater
evaluation programs, Harris and Schaubroeck (1988) cite a variety
of advantages that have been associated with such programs in the
past. These advantages include an enhanced ability to observe
and measure different aspects of performance, greater reliability
of the ratings, heightened perceptions of fairness and acceptance
by those rated, and a more defensible legal foundation for an
organization's overall evaluation program.

A number of investigations have explored in some detail the
differences in perspective between superior, peer, and
subordinate raters (Cann & Siegfried, 1987, 1990; Halpin, 1953;
Harris & Hogan, 1992; Moore & Smith, 1952; Mount, 1984). Moore
and Smith (1952) interviewed U.S. Air Force personnel about the
characteristics of the best and worst noncommissioned officers
they had known. The percentage of individuals mentioning various
characteristics differed among groups of commissioned officers
(superiors), noncommissioned officers themselves (peers), and
airmen (subordinates). For example, superiors frequently
mentioned that a good noncommissioned officer does not require
continuous supervision, is a disciplinarian, and is not
arbitrary. In contrast, peers often mentioned that a good
noncommissioned officer sets the example, commands respect, and
provides leadership. Subordinates frequently said a good
noncommissioned officer issues quiet and authoritative commands,
looks out for the welfare of subordinates, and respects
subordinates as people. Despite these differences, good
noncommissioned officers were often viewed as knowing their job
and having a neat appearance by those in every group. Moore and
Smith (1952) concluded that one's views about leadership "depend
upon one's position in a leadership hierarchy" (p. 5).

In a study of B-29 aircraft commanders flying combat
missions over Korea, Halpin (1953) found that superiors tended to
rate favorably those commanders who were described by their
subordinates as being high on the initiating structure dimension
of leader behavior. However, subordinates tended to rate
favorably those commanders they found to be high on the



REPORT DOCUMENTATION PAGE

1, REPORT DATE 2. REPORT TYPE 3. DATES COVERED (from. . . to)
1996, April Final March 1994-May 1995
4, TITLE AND SUBTITLE Sa. CONTRACT OR GRANT NUMBER
An Exploration of the Construct Validity of a Leadership Behavior 5b. PROGRAM ELEMENT NUMBER
Rating System 0602785A,
6. AUTHOR(S) 5c, PROJECT NUMBER

A791
Elisa H. Schwager and Kenneth L. Evans 5d. TASK NUMBER

1115

Se. WORK UNIT NUMBER

Hol
7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 8. PERFORMING ORGANIZATION REPORT NUMBER

U.S. Army Research Institute for the Behavioral and Social Sciences
ATTN: PERI-RM

5001 Eisenhower Avenue

Alexandria, VA 22333-5600

9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. MONITOR ACRONYM

ARI
U.S. Army Research Institute for the Behavioral and Social Sciences

5001 Eisenhower Avenue 41. MONITOR REPORT NUMBER
Alexandria, VA 22333-5600

Technical Report 1041

12. DISTRIBUTION/AVAILABILITY STATEMENT

Approved for public release: distribution is unlimited.

13. SUPPLEMENTARY NOTES

14, ABSTRACT (Maximum 200 words):

The construct validity of a leadership behavior rating system used by the U.S. Military Academy (USMA) was examined as a prelude
to a longitudinal program of research on leadership development. Their system centers on Cadet Performance Report (CPR) ratings
made by superior, peer, and subordinate raters. CPR ratings include individual scores on 12 leadership behavior dimensions and a
global score of performance. Two CPR dimensions, duty motivation and military bearing, were found to be most strongly related to
the CPR global score and the leadership grade, which is USMA’s official evaluation of a cadet’s overall performance in a leadership
role. Although this finding was consistent for each type of rater, the relationship of other dimensions to the global score and
leadership grade suggested different types of raters focused on different dimensions when evaluating overall leadership performance.
Additionally, four broader factors were found to underlie the 12 CPR dimensions, based on a synthesis of the results from 19 principal
components analyses. Consistent with other leadership behavior taxonomies and other research concerning the effects of rater position
on the leadership evaluation process, our findings offer some evidence of the construct validity of the CPR dimensions for measuring
cadet leadership performance.

15. SUBJECT TERMS.

Leadership Leadership measurement Leadership evaluation
SECURITY CLASSIFICATION OF 19, LIMITATION OF | 20. NUMBER | 21. RESPONSIBLE PERSON
ABSTRACT OF PAGES (Name and Telephone Number)

16. REPORT 17, ABSTRACT 18. THIS PAGE <
Unclassified | Unclassified | Unclassified | Unlimited 52



rated cadets. It is apparent that duty motivation was the most
frequently used dimension by each type of CPR rater. Military
bearing was another often used dimension, especially among the
three different types of cadet raters. However, differences in
usage were found along most of the other ten dimensions.

With respect to cadet raters, superiors were more likely
than either peers or subordinates to use the dimension of
supervising. In addition, superiors were more likely than
subordinates to use the dimension of planning and organizing, as
well as the dimension of oral and written communication. In
contrast, both peers and subordinates were more likely than
superiors to use the dimension of consideration for others.
Further, subordinates were more likely than either superiors or
peers to use the dimensions of influencing others and developing
subordinates. Of the three types of cadet raters it appeared
that superiors and subordinates had the most distinctly different
patterns of dimension usage, with peers being similar to
superiors on some dimensions and similar to subordinates on
others. Although peers may have had a perspective about cadet
leadership performance that was less distinctive, a comparison of
dimension ranks in Table 9 suggests peers placed a relatively
greater emphasis on teamwork and professional ethics than either
superiors or subordinates.

Compared with cadet raters, instructors were more likely to
use the dimensions of decision making and oral and written
communication, although they were somewhat less likely to use the
military bearing dimension. Additionally, it was found that many
instructors did not rate cadets at all on the dimensions of
delegating, supervising, and developing subordinates. This
resulted in lower rates of usage for these dimensions, relative
to the other nine. In all probability the classroom environment
affords less opportunity to observe leadership behavior in these
latter three dimensions, even when cadets in the upper classes
are being observed. As many courses are composed of cadets
within the same class year, opportunities to lead cadets in the
lower classes take place largely outside of the classroom.

27



Table 3

Means and Standard Deviations of CPR Dimension Scores and the
CPR Global Score for Superior, Peer, and Subordinate Raters

Superior Peer Subordinate
Dimension M sD M SD M sD
Duty Motivation 4.18 .80 3.85 .90 3.77 .78
Military Bearing 3.79 .84 3.59 .81 3.53 .76
Teamwork 3.57 .75 3.44 .80 3.33 .63
Influencing Others 3.25 .56 3.19 .60 3.25 .67
Consideration
for Others 3.32 .65 3.37 .76 3.36 .70
Professional Ethics 3.28 .62 3.32 .62 3.28 .59
Planning and
Organizing 3.40 .74 3.29 .65 3.22 .55
Delegating 3.14 .43 3.11 .41 3.12 .41
Supervising 3.21 .55 3.12 .43 3.15 .50
Developing
Subordinates 3.31 .64 3.19 .53 3.33 .72
Decision Making 3.13 .49 3.13 .47 3.09 .37
Oral and Written
Communication 3.39 .69 3.24 .60 3.21 .52
Global Score 4.25 .74 3.97 .91 3.94 .86

Note. Depending on the dimension, there were between 2013 and
2375 cadets rated by superiors, between 2042 and 2238 rated by
peers, and between 1221 and 1294 rated by subordinates.

15



U.S. ARMY RESEARCH INSTITUTE
FOR THE BEHAVIORAL AND SOCIAL SCIENCES

A Field Operating Agency Under the Jurisdiction
of the Deputy Chief of Staff for Personnel

EDGAR M. JOHNSON
Director
eS
Technical review by
Robert J. Lamb

Joel M. Savell

NOTICES

tion of this report has been made by J. Please ai
arch Institute for the
-STP, 5001 EisenhOwer Ave., Alexandria, Virginia

Behavioral a
22333-5600.

Sciences, ATTN: P
FINAL DISPOSITION: This report may be destroyed when it is no longer needed. Please do not
return it to the U.S. Army Research Institute for the Behavioral and Social Sciences.

NOTE: The findings in this report are not to be construed as an official Department of the Army
position, unless so designated by other authorized documents.



g. PLANNING AND ORGANIZING: The ability to establish a course of action for oneself and others to accomplish goals; establishing

Priorities and planning appropriate allocation of time and resources and proper assignment of people

h. DELEGATING: The ability and inclination to use the talents of subordinates effectively; the allocation of decision-making and other
authonty to the appropriate subordinates,

i. SUPERVISING: Tne ability to establish procedures for monitoring and regulating processes, tasks, or activities of subordinates and one's own
job; taking actions to monitor the results of delegated tasks or projects.

j. DEVELOPING SUBORDINATES: The art of developing the competence and self-confidence of subordinates through role modeling

and training and developmental activities related to their current or future duties.

k. DECISION-MAKING: The ability to reach sound, logical conclusions based on analysis of factual information and the readiness to take
appropriate actions based on the conclusions.

1. ORAL AND WRITTEN COMMUNICATION: The ability to express oneself effectively in individual and group situations,

either orally or in writing, includes utilizing proper grammar, gestures, and non-verbal communication.

04 07042 O02 O02 O02 Orlpz ga

PART IV - OVERALL PERFORMANCE AND POTENTIAL (Rater)

a. Compared to all others I have seen in this ’. General comments on performance and overall potential.
duty position, this cadet ranks in the
RANKING (check one)

UPPER
10%

UPPER

25% ¢. Recommendations for improvement,

MIDOLE|
30%

LOWER
25%

LoweR|
10%

PART V - OFFICER REVIEWER

ta fdo G2 donot) concur with the | © General comments on performance, potential, and recommendations for improvement.

rater’s evaluation

Return original to: Commandant of Cadets, USCC. ATTN: MACC-O-SP-LD, West Point, New York 10996

USMA Form 2-543R (Oct 93)



communication) had relatively higher loadings on the second
component than on the first. Here, the loadings of the 12
dimensions on the second component were used in calculating
median loadings for both the self-management component and the
cognitive component. Finally, no loadings from the analysis were
used in calculating median loadings for the informal
interpersonal component, because there was no clear difference in
the loadings of the remaining dimensions across the first and
second components. We anticipated that a given dimension's
highest median loading would be on the one component hypothesized
to underlie that dimension. Yet, because some sets of loadings
were used in calculating the median loadings of more than one
component (as in the example just outlined), we realized these
procedures would produce rather conservative estimates of the
actual conceptual differences existing among the components.

Table 8 shows the median loadings of each CPR dimension on
the four components. Six dimensions had a median loading on the
hypothesized component that was substantially greater than on any
of the other three components. Differences between the two
components having the highest median loadings ranged from .21 to
.47 on these six dimensions. Further, such differences ranged
from .07 to .13 on four other dimensions. However, the median
loadings for the dimensions of influencing others and
professional ethics varied little across three of the four
components. Because of the unanticipated results encountered on
these two dimensions, we individually examined the 19 principal
components analyses for an explanation. In doing so we
discovered that the influencing others and professional ethics
dimensions emerged less often than the teamwork and consideration
for others dimensions as part of the informal interpersonal
component.

An alternative way of examining Table 8 is to focus on the
components separately. Looking down each of the four columns it
becomes evident that the dimensions we hypothesized to underlie a
particular component did in fact receive median loadings that
were higher than those dimensions not hypothesized to underlie
the component. The only exception was that the decision making
dimension had a higher median loading on the informal
interpersonal component than was expected.

Overall, the results of the principal components analyses
suggested the CPR dimensions to be interrelated, as reflected in
the emergence of four broader factors of leadership behavior.
Because dimension scores within a component tend to be more alike
than dimension scores across components, one can develop many
expectations about the nature of future CPR ratings (e.g., that
military bearing scores will be more like duty motivation scores
than delegating scores). Although simpler factor structures
could have been advanced for rather limited purposes (e.g.,
understanding the nature of First Class peer ratings), the four-
component structure we hypothesized seems appropriate for more
general use in understanding how the CPR dimensions relate to one

23



Although the nature of our data precluded the calculation of
meaningful reliability coefficients (i.e., cadets had only one
superior rater and cadets with presumably moderate performance
had no peer, subordinate, or instructor raters), we observed
several trends concerning CPR reliability in the process of
averaging across raters for cadets with multiple peer,
subordinate, and instructor ratings. For every type of rater,
the most reliable part of the CPR appeared to be the global score
of performance. Raters generally agreed about whether a
particular cadet was a high or low performer. However, a
consensus was reached less often in describing the nature of a
cadet's high or low performance. Thus, CPR dimension scores
tended to vary more across raters than the global score, though
duty motivation and military bearing scores seemed less variable
than scores on the other dimensions. The least reliable part of
the CPR appeared to be the selection of dimensional strengths and
weaknesses. Compared to the dimension use percentages of
superiors, the degree to which raters disagreed about a cadet's
relative strengths and weaknesses was reflected in lower reported
dimension use percentages for peer, subordinate, and instructor
raters (as was shown in Table 9).

Even though describing large groups of cadets in terms of
their strengths and weakness can be informative (such as to
explore differences among classes or differences among types of
raters), caution should be used in making judgments about the
strengths and weaknesses of an individual cadet. Because it is
not unusual for raters to disagree about an individual cadet's
strengths and weaknesses, one should be especially careful in
cases where strength and weakness judgments come from one rater.
Only when there is clear agreement among raters can one have
confidence in the accuracy of this type of information. Because
they represent the combined judgments of multiple raters,
averaged CPR ratings are apt to be more reliable and valid than
those based on a single rater.

In exploring the construct validity of the CPR dimensions we
also learned more about the nature of the leadership grade as a
holistic performance measure. In terms of the CPR dimensions,
the leadership grade appears to measure cadet performance
primarily in the areas of duty motivation and military bearing.
Yet, these dimensions accounted for only about 15% of the
variance in cadet leadership grades across classes. Because most
of the leadership grade variance is unexplained in terms of CPR
dimension scores, there is much about the leadership grade that
we do not understand at present. However, we are aware from some
of our other research that duty position is an important factor
to consider in predicting the leadership grades of First and
Second Class cadets, though duty position appears to be much less
important in predicting the CPR global scores of these same
cadets. It is not yet clear why the leadership grade and the CPR
global score differ in this way. In order to resolve issues like
this one, one goal of our future research will be to determine

32



Table 5

Summary of Stepwise Multiple Regression Analyses Predicting the
Leadership Grade From CPR Dimension Scores for Superior, Peer,
and Subordinate Raters

Dimension AR? R?
Superior
Duty Motivation 13 -13
Military Bearing 02 15
Planning and Organizing 01 16
Peer
Duty Motivation 211 11
Military Bearing +03 14
Planning and Organizing O01 15
Subordinate
Duty Motivation 13 -13
Military Bearing -02 -15
Consideration for Others -O1 +16

Note. Regression analyses were based on 1920 cadets for
superior raters, 1929 for peer raters, and 1150 for subordinate
raters. The incremental variance explained by each of the above
dimensions was statistically significant, p < .01.

18



The increased acknowledgment that leaders are problem
solvers who act upon organizational systems has become evident in
many of the dimensions found in more recent leadership
classification schemes (see Fleishman et al., 1991). For
example, the existence of planning and decision making dimensions
in a variety of leadership taxonomies attends to the cognitive
requirements of leadership. Viewing the leader as a problem
solver implies comparability of leadership requirements across
settings and allows for greater understanding of effective
organizational leadership behavior in a wide variety of contexts
(Mumford, 1986). In fact, Fleishman et al. (1991) suggest that
"leadership behavior represents a form of organizationally-based
problem solving, implemented in a social context, where an
attempt is made to bring about goal attainment by influencing the
actions of other subsystems" (pp. 258~-259).

Leadership Dimensions in the U.S. Army

The U.S. Army has conducted a considerable amount of
research on defining and measuring leadership effectiveness.
Three classification systems are particularly relevant for
understanding leadership in the Army context. The Center for
Army Leadership (CAL) has identified nine leadership competencies
representing the performance requirements of leaders throughout
the U.S. Army. Two other systems were developed to classify the
leadership behaviors involved in precommissioning education and
training. They are the Leadership Assessment Program (LAP), a
sixteen-dimension taxonomy used in the Reserve Officers Training
Corps (ROTC), and the Cadet Performance Report (CPR), a twelve-
dimension leadership behavior rating system employed by the U.S.
Military Academy (USMA) at West Point, NY.

The nine CAL leadership competencies are communications,
supervision, teaching and counseling, soldier and team
development, technical and tactical proficiency, decision making,
planning, use of available systems, and professional ethics
(Department of the Army, 1990). The CAL competencies were
originally established to provide a broad framework for assessing
Army leadership performance (Clement & Ayres, 1976). The
identified leadership competencies were viewed as reflecting the
requirements faced by leaders Army-wide.

Support has been found for the CAL competencies as
representing a meaningful classification system of leadership
behavior (Tremble, 1992; Twohig & Tremble, 1991). Specifically,
these studies found relationships in tactical training
environments between the identified competencies and leadership
quality, leader effectiveness, and unit effectiveness. In
relation to these competencies, some researchers also have argued
for tailoring leader assessment programs to the requirements of
leadership positions at different organizational levels (Clement
& Ayers, 1976; Twohig & Tremble, 1991).



Appendix B

An Example (Fictional) of a Completed CPR
from the Automated System



personal composure under stressful conditions. These
similarities between the LAP and CPR may be due to the fact that
they were both created to assess performance in precommissioning
education and training environments. Certainly, each system has
evidenced a degree of content validity for that purpose (OIR,
1989; Rogers et al., 1982).

Interestingly, the use of available systems dimension in the
CAL classification system is not present in either the LAP or
CPR. Exclusion of this dimension may be attributed to the
central role that computer and information management technology
have in contemporary post-secondary education. Because most ROTC
and USMA cadets already display considerable familiarity and
facility with such technology, particularly as they approach
graduation, it may be viewed as a less important dimension upon
which to differentiate leadership performance in those
populations.

Overall, the three Army taxonomies appear to incorporate
both specific dimensions representing the military leader role,
as well as other dimensions having greater generalizability
across organizations. In fact, other researchers have
incorporated many of these actual dimensions in their broader
categorizations of leadership behavior, which are not limited
solely to military organizations (Fleishman et al., 1991;
Korotkin, Mumford, Yarkin-Levin, Wallis, & Fleishman, 1986).
While the overlap among the dimensions shown in Table 1 lends
some conceptual support to the belief that they are somewhat
representative of the leadership domain, additional empirical
evidence is needed before one can conclude that measures of these
dimensions reflect actual leadership performance, either
holistically or componentially. This is particularly true of the
CPR dimensions, which were developed most recently.

Effects of Rater Position on the Evaluation Process

While the meaning and interrelatedness of leadership
behavior dimensions must be considered when evaluating their
construct validity, the importance of a particular dimension in
evaluating overall leadership effectiveness is often dependent
upon the perspective of the individual providing the rating.
Superior, peer, and subordinate raters may place differential
significance on the dimensions contributing to their overall
leadership assessment. Thus, the effects of the positional
relationship between rater and ratee on the evaluation process
must also be considered when examining the construct validity of
a leadership behavior rating system.

While superiors have been used to provide evaluations of
leadership performance most often, other studies have indicated
that peers or subordinates can also provide evaluations having
sufficient levels of reliability and validity (Hollander, 1957,
1965; McEvoy & Beatty, 1989; Shipper & Wilson, 1991). In fact,