-~ - —r

Technical Paper 368

!. ‘

~~~~~~

‘ CONSTRUCT VALIDITY OF LEADER

EFFECTIVENESS CRITERIA

Ronald G. Downey, Paul J. Duffy,

- and Samuel Shiflett

PERSONNEL AND MANPOWER TECHNICAL AREA

I

1: DDC

~

ir

p 

Resear

U. S. Army

~ ~h lnstitut

~ 

Vor the Rch wiorat and St~

dal Sciences

June 1979

7 9 10 22 072

Appr ov•d for Public rilis s.; d i5tri b uti o~ unl imited .

I ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ —

—- ~~~~~~~~~~~~ -- --
—

~~~

—~~

U. S. ARMY RESEARCH INSTITUTE

FOR THE BEHAVIORAL AND SOCIAL SCIENCES

A Field Operating Agency under the Jurisdiction of the

Deputy Chief of Staff for Personnel

WILLIAM L. HAUSER

JOSEPH ZEIDNER Colonel , U S Army T•chn ical Director Commander

NOTICES

OISY RISUTION Primsry distribinign of this ripen Piee bun m.d. by A’l. PtSaS, addiess CO $p0~dSfl~

S Concerning disivlb ~

ñio,~ Of SPOilS to: U. S. Army ~sIes’ch Institute for the Beltiviersi end Seciel Sciences. ATTN. PERI.P, 5001 Eis’nho~

% r Avenue. AII*i

~drji Virginia 22333.

FINAL 01j’°’1TIQtI Thi, ~~~~ ‘n.y be destroyed when it is no longer needed. PISSR do not return it to t

~e U. $ Army ~ssearch Institute for thu Sshsw ,o,.i end Social Sciences.

~~~ 

Thu finding, in this rspe,~ .1. not to be constr~

ai

~ a. an off icl.I O.ocnmsnt of tI’s Anny position . unhuis a. desians’ed by Othor eutP,Oc,fld document,.
Unclassified SECURITY CLASSiFiCATION OP THIS PAGE (lPli .n 0.1. Enl.~.d) ___________________________________

~~~~~~ 

DOC””E ~~~~~~~~~~~~~ 

READ INSTRUCTIONS 5% r’Ji%I UM ~~ I 

~~ I 

~~~~ BEFORE COMPLETING FORM

I

” ~~~ ~~~~~~~~ —s 2. GOVT A CCESSION NO 3. RECIPIENT S CATALOG NUMBER

(9 Technical Paper,~)68 I _____________________________

• “—“~ L~~~~~t.B ~~~~~~~~~ tj 5. TYPE OF REPORT & PERiOD COVERED

• 

. 

~

4” ~~~~STRUCT VALIDITY OF~~~~ADER~~~~FECTIVEN~~ 8

~ --

1* CRITERIA — — ___________________________

\

~~

,_, ~~~~,. 6. PERFORMING ORG. REPORT NUMBER

A ~~~~~~~~~~~~~~~~~~ Paul ~ g~~~ ffYi~~~~ - 

--

~~~~~~~~ 7. AUTH •. .-• —— 5. CONTRACT OR GRANT NUMBER(I)

(.1 ~~~muelJ~~~iTI~

ett _ ~~ - -

5. PERFORMiNG ORGANIZATION NAM E AND ADDRESS 10. PROGRAM ELEMENT. PROJECT . TASI(

U.S. Army Research Institute for the Behavioral ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

and Social Sciences .

5001 Eisenhower Avenue , F lexandria , VA 2233I

~ 2Ql627l7A766~~~~

II. CONTROLLING OFFICE NAME AND ADDRESS ~iZ. .uri~ ORT’f -

Deputy Chief of Staff for Personnel Jun~~~~ 79 ‘

Washington , DC 20310 ‘. 4uuern .riA~~~~~~ T

54. MONITORING AGENCY NAMC & ADDRESS(SI dlli.,wt Iron’ Contr of lf nd OWc.) 15. SECURITY CLASS. (of thi. r.porl)

-- 1 

~ ~~~ 

2 Unclassified

IS.. DECLA SSIFICATION/~~~WNGRADIN O

IS. DISTRIBUTION STATEMENT (of IbIs Rspo,t)

Approved for public release ; distribution unlimited.

17. DISTRIBUTION STATEMENT (of A. .b.1r.cl .fltl ’sd in Block 70, If dItf.,.nt from R.porl)

-- AlI.

~~—3U

IS. SUPPLEMENTARY NOTES

IS. KEY WORDS (Contlnia. on r.v. ,.. old. If n.c ...~~~ ond Id.ntlfy by block nionb.r)

Leadership effectiveness

Analysis of leadership criteria

20. AUS

’

~~~AC1. (c ru .. ~~ ,.~~

... .

~

d. If ~~~~~~~~ d sd.nIsfr b~ block nomb. r)

Previous research on leadership effectiveness has often used cr iter ia

of one type and from one source. In this research , data on leadership effectiveness were collected with a variety of measures from different sources

(leaders, peers, subordinates). Findings were analyzed for statistical

convergence and by a multitrait-multimethod intercorrelation matrix-. A

satisfying degree of convergence but relatively little divergence appeared ,

and 47% of the variance was not accounted for by either one . Apparently,

(Continued)

t.J ~~~~7) 1Q3 £D~

TIowoF lwOv SS IS OBsoi.ETE Unclassif ied

L I ~~~~~~~~ CL. ASSIFICATt ON OP THIS PAGE (~~ sil 0.1 . SiN
-- -•-~—— -.

~~--— •~---- __~~

__ ___

~~w,~~~~~~~ • • ~~

—------

~~

Unclassified

SECURITY CLAUIPICATIOII OF THIS PAGE(~~ on Dol. Ia..d ~

Item 20 (Continued)

many measurement techniques or sources of data are essentially interchangeable . One cannot safely interpret results across different studies, however , because of the large amount of variance that remains unaccounted for

by either convergent or divergent validity . The report is written for

psychologists.

ACCESSION for

NTIS White Section

DOC Butt Section o

UNANNOUNCED 0

JUSTIFICATION _____________

BY •. •

DISTRIBV11~IAYAILABflJTY cOOES

• 01st. AVAIL and/or SPECIAL

Unclassified

jj SECURITY CLASSIFICATION OF THIS PAGE(WSi.n D.1s Snl.r.d )
- — - - ~~~~ -~~—- -

— -

~~ • ~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~

I

Technical Paper 368

CONSTRUCT VALIDITY OF LEADER

EFFECTIVENESS CRITERIA

Ronald G. Downey , Paul J. Duffy ,

and Samuel Shiflett

• 

• Submitted by:

Ralph R. Canter, Chief

PERSONNEL AND MANPOWER TECHNICAL AREA

Approved By:

E. Ralph Dusek

PERSONNEL AND TRAINING

RESEARCH LABORATORY

Joseph Zeidncr

TECHNICAL DIRECTOR

( U.S. ARMY RESEARCH INSTITUTE FOR THE BEHAVIORAL AND SOCIAL SCIENCES

• 5001 Eisenhower Avenue, Alexandria, Virginia 22333

• Office , Deputy Chief of Staff for Personnel

Department of the Army

June 1979

• Army Project NUmber Leader Effectiveness

2016211 7A766 Criteria

pprose~ for pu~

iic reu se; distribution unIhi~tSd.

1

~~

}

~~• 

----•—---—

— — •- —~--

~--‘ — --—.—.--—-— .~~-. — —.. - ~~~~~~~~~~~ 

~~~~~~~~ _d ~~~~~~~~~~~
- 

~~~—•— -— —-—-• 

ARI Research Reports and Technical Papers are intended for sponsors of

R&D tasks and other research and military agencies. Any findings ready for

implementation at the time of publication are presented in the latter part of

the Brief. Upon cornp)etion of a major phase of the task, formal recornmendations for offkial action normally are conveyed to appropriate military

agencies by briefing or Disposition Form.

4

___________________________ 

iv 

_____ 

______________

-a ~~~~~~~~~~~~~~~~~~~~~ 

~~~~~~~~~~~~ _ — - -
• -- • ~~~~~~~~~~~~~~~~ 

••

FOREb~)RD

This research , carried out in the Personnel and Manpower Technical

Area of the Army Research Institute for the Behavioral and Social Sci-

• ences (ARI), evaluated several modes of estimating leader effectiveness

• for comparability of the data produced by each .

The research was an in—house effort, responsive to Army Project

2Q162fl7A766 and to special requirements of the Office of the Deputy

Chief of Staff for Personnel. At the time the research was conducted ,

both Samuel Shiflett and Ronald C. Downey were part of the API research

staff. Dr. Shiflett is now at New York University, New York City, and

Dr. Downey at Kansas State University , Manhattan , Kans. Robert Sulzen

helped arrange and coordinate the data collection and provided insights

into the group processes during the exercise. Frank E. Saal gave helpful conanents on the paper.

SEPH ZE NER

Technical Director

V

• iN

• • •, ---••• •

~~ ..-•‘•••—• -..-—-— •—

— ~~~ _ — ——— —
CONSTRUCT VALIDITY OF LEADER EFFECTIVENESS CRITERIA

BRIEF

I

~quirement :

To investigate whether different measures of leadership effectiveness from different so’

urces provide comparable data.

Procedure :

Data on leader effectiveness were gathered during a 2—week field

exercise from three different sources (leaders, peers, and subordinates), using several different techniques (questionnaires on leader

performance, leader self—ratings, and peer evaluation of leader effectiveness). These data were analyzed in a multitrait-multimethod

fashion.

Findings:

The rating situation and raters’ perceptions interact with leaders’

behavior and evaluations. Most types of measuring techniques seemed to

be evaluating the same basic variables, with the exception of peer

evaluation of leadership potential. Exact amounts of interchangeability

could not be determined .

Utilization of Findings :

The data suggested that different methods of measuring leadership

may be subject to different kinds of bias, and that using more than one

method and source in the assessment may be essential in measuring all

• aspects of leader performance.

vii

-•• • • • ~~••

~~~~~~~~~ • - •• • ~~ •~~~~~~~~~~~~~~ •~~~~~~~~~~
-I’

CONSTRUCT VALIDITY OF LEADER EFFECTIVENESS CRITERIA

CONTENTS

Page

INTRODUCTION 1

NETHOD 2

• Subjects 2

• •

• Procedure 2

RESULTS 2

DISCUSSION 6

REFERENCES 9

DISTRIBUTION 11

LIST OF TABLES

Table 1. Item content of leader effectiveness, group

performance , and group relations measures 3

2. Intercorrelations of different methods of measuring

leader effectiveness 4

3. Multitrait—multimethod matrix of measures of leader

effectiveness , group performance , and group relations 5

4. Analysis of variance of correlations 7

ix j
—•~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1

~~~~

CONSTRUCT VALIDITY OF LEADER EFFECTIVENESS CRITERIA

INTRODUCTION

Research over the past several decades has examined leader effectiveness increasingly in terms of the leader’s interaction with the

group and the resulting impact upon outcomes (Stogdill , 1974). Regardless of the researcher ’s theoretical stance, the definition of

leader effectiveness usually focuses on either the leader’s behavior

or the leader’s effect upon group processes or outcomes.

An outgrowth of this situation has been a proliferation of measurement techniques and approaches for assessing the effectiveness of leaders. Typical measurement techniques have included trait ratings, behavior descriptions or perceptions, and personality measures. In

addition , researchers often gather data from different sources, including superiors, outside observers, subordinates, or the leaders themselves. Unfortunately , this abundance of methods has resulted in a

confusing collection of sometimes uninterpretable and even contradictory

findings. The crux of the problem is the attempt to establish the

construct validity of the criteria used in leadership research.

One technique for reducing this confusion is the multitrait—

multimethod (MT/MM) correlation matrix for examining the convergent

and discriminant “alidity of several measures from several sources

• (Campbell & Fiske , 1959) . Although this technique has been used in

several different areas, it has not been used extensively in the leadership area. Yunker and Runt (1976) used this technique to compare

behavioral descriptions of leader effectiveness; and Lawler (1967)

and I(avanagh, MacKinney, and Wolins (1971) used it to examine managerial performance ratings from three different sources.

The Kavanagh et al. (1971) study is particularly interesting because it suggests an analysis of variance technique that simplifies

the analysis and interpretation of a multitrait-multimethod matrix .

The ANOVA technique was applied to 20 ratings from each of three different respondent sources to demonstrate that it provided at least as

much information as the cumbersome interpretive requirements of a

60 x 60 matrix.

This study adopted this analysis of variance technique to augment

the traditional MT/MM approach and to simplify interpretation of the • matrix . Specifically, this study investigated the degree of convergence and divergence among various techniques for evaluating leader

effectiveness and utilizing different sources for these evaluations.

1

• _ _

- ~~~~ g • ~~~~~~~~~~~~~~~~
METHOD

Subjects

A 2—week field training exercise for an Army Special Forces Reserve Unit was the setting for this investigation. Data were gathered • from 275 reserve unit members from 23 detachments; each detachment

contained 10 to 12 enlisted specialists, an executive officer, and a

commanding officer. The exercise scenario required each detachment to

train another group of reservists in guerrilla—type activities. Data

also were gathered from 346 other reservists from the 23 detachments,

• herein designated as participant observers. Thus, data were collected

from three different sources: leaders, group members, and participant

observers.

Procedure

Questionnaires were administered at both the beginning and the

end of the exercise. All data reported in this report were gathered

at the completion of the exercise. Scales administered to the leaders

and group members included the Military Leadership Behavior Survey

(Downey, 1974), which is similar in methodology and content to the

Leader Behavior Description Questionnaire (Stodgill, 1963). The 53—

item Military Leadership Behavior Survey provides scores on the follow—

ing scales: task professionalism , task-oriented consideration , peopleoriented consideration , and personal/interpersonal professionalism .

In addition , each leader estimated his own performance on a 5-point

scale. Also, group members nominated the three peer group members

(including the formal leaders) in terms of highest leadership potential.

Scores were derived from questionnaire items about exercise tasks

for group and leader ratings of leader effectiveness , group performance,

and group relations (Downey , Duffy, & Shiflett, 1975). The questionnaires completed by the participant observers included the same exercise—

related items. Although there were differences in the specific items

defining the factors across groups (Downey et al., 1975), scores for

leader effectiveness, group performance , and group relations were also

derived for the participant observers. Table 1 shows the number of

items in each scale for the internal and external sources and the number of conlnon items .

RESULTS

The analysis proceeded in t1.ree stages. The first step investigated the degree of convergence among measures directed specifically

at the leader’s behavior . The data necessary for assessing discrimination among all the leader behavior measures were not available for

all three sources; hence , the first step in the analysis examined 

only2
the degree of convergence among all the leader effectiveness (direct • estimates) measures and from all sources, whether or not a particular

variable was available from more than one source. Table 2 shows the

• correlations among all these measures. The leadership potential measure in Table 2 is the peer evaluation score derived for the formal

leaders. The data showed high convergence among all of the measures

from the group members, which, as demonstrated by Weissenberg and

• Kavanagh (1972), is not uncommon for measures of this type. The leader

effectiveness measure obtained from the participant observers also

showed ~ fair degree of convergence with the measures from the group

members, as well as the leaders’ ratings of their own effectiveness.

This was the only instance in which the leaders’ self-ratings were related to any of the other leader effectiveness measures. The peer • evaluations of leadership potential showed little if any relationship

to any of the other variables.

Table 1

• Item Content of Leader Effectiveness, Group Performance ,

and Group Relations Measures

Leaders and Participant

Variable detachment members observers

Leader effectiveness 6

a 4

Group performance 17 7

Group relations 6 6

a

The leader effectiveness measure for the formal leader was a single

item.

The second step of the analysis developed a multitrait—multimethod

intercorrelation matrix (Campbell & Fiske, 1959) with the leader effectiveness, group performance, and group relations measures from each

of the three sources. This analysis revealed the extent to which the

measures were converging or diverging and, consequently, determined

whether or not they could be used interchangeably. Table 3 presents

these results. Following the guidelines prescribed by Campbell and

Fiske , the results showed that for the criterion of convergence——the

significance of correlations in the validity diagonals-—five of the

nine correlations were significant. In general , there was more convergence for the leader effectiveness and group performance measures

than for the group relations measure.

3
~

••

•—

~

-.~

. 

—

• 14 N ~-4 ~~ r~

4 I 

5

~~ ~~ N LI) 

I

S

U)

.0

r- N a’ ~~ 

r’

~ I 

0

N N 0 I 4.)

040 N N II) N N I N N ~ 4 C’) N I 4.3

14

‘U

• 

.4, • • 04

4, .4, II It) a’ LI) I

S N I 0

0 .0 

-

.~

4 ‘U 14 14 •..I • 14 .4, • ‘O U) iU~~ a’ a’ 0 I ‘U ‘U a’ I S U ‘4.40 — .4,0 SN a’ I

‘0

OIl 14 • .0 5 • I

~~~Z .4, 4))

04 C’.1 N I U) a

~ I 

II

1-)

U)

• N O W 04 s o 4)) 1 45 -

.-4 W~~~ 04 .0 *4.4 ..

4 

-

~4 (4) (4.4 4) I 

0

E~ •..

I U 14

0’

(4.4 94

o w

0

05 

4)

O ’O 

0S

4.3 4)) ‘o 

w 0 0 0 0 0 0 04 I-

~ 0 

4)

0 4.3 S 04 14 4)) 0

0 

0 F S 0) 0)

.0

H .—4 

0) (U • E 0 5 .-4 U) U) 44) (0 0 U) ‘U 4)) U) .1 4)) 5 ..4 4)) 5

‘-4 14 0 .4.1 ~ (0 ‘

U 0) 0) 0 0) 5 

4.)

0 ‘0 (.1~

~5 ~ 0) > > 

0 0 5 14(0 .~4 43 •..4 ..4

‘0~~~ 4.

3

~~~ W .

~

.4 4) 0 

4.3 

4.3 (0 0)0 00 4J e4 4.) 0

~ 0 

0 4)) .4J .,4 5 •-I 044 ) 0) 0) 0) 0) 04J .4 4J .,4~~~ 94 04 ~4.4 U.4

‘I-I S ‘U 14 ‘U ~~ 0 

~1-I 

•.4 44.4 44.4 • 0 .414 0 14 ‘-4 —4 0) .0 0) 0) 14 145 101 ‘U S (4) 

.44

04 

O’0 

W ’0 04)1 14 14 14 14 C.) 144 .4 •.4 OW S S S 0) . . 4 (

.Q V V V V ~~ 0 0

.Q ‘U 44)0 0)0 0)14 0) 4)) 5 4))

• 

‘

U I’ 

~4U 040 040~ ,.

~ ,.~ • v v

.44 0) 14 4.3 04O

~

4

_________ - 

-

-*

-

.-

• - ~~-~~~~

---
_______________________________ ______________________________________________ - •.•— --. •

~~ • •—••.

C’)

P1

(‘4 IIfl 04 I N

ION 04 ~~~~~~~~

U)

U) • 

~•

. 

0)

0 1’) I .fl~~~~~N I O N O .44 I

4.3 5

• 

• 0—54 4) .0

‘UN I

~~~Q • 444 .,.~~~~~ I N

liZ 0)

‘0 • ‘U O) .44 (.4 0 I

~~~~~ m

.44

‘

4441

O ’ U

(4) • • I ~~~ -4N O a ’C ’ 0

(4~ U)~~~ ‘UO 0) 5 14 r.4 • ZO (.4 (‘)~~~~.44

0 I

~~~~ (‘444.4

E-’

14 01 .-4 ION ~~~ 0 N 0 I(4) C’) NON

0

4.1 14

.44

U

4)04

~ 0 .-4 N (4) .44 N (4) .44 N (4)

0 00

.1.3

--4

‘U14 U) U) 4)1 4.) 4)) 4)) 4)) .44

4’ 001 O W U) O W ‘.4 W1 ) 0 )4.1 14 0 ) 0 > 0(0 >04 )) 0) > 04)) Z • . - I’ U O -44 44) 0 > ~

. 4’UO 4)~~~ 0 14 4J~~~~O 0 14. 44 014-44 0) 014-44 0) 041 0 ) 0 4 .3 4)) 0 1 0 4 .3 ‘I-4 ’l-4 ’U .0 9414 .-i 9414 .-f 0 941 4 .4

0)010) 0)50) 5010) 4) 0414

.5: 1 ~~~~~~~~~~~~~

~~~~~~~~~~ 

~ ~~~~~~~ -

~~ ~~~~~~~ v v

S S S 

,

~~ 

• . • 

•

~~ . . . .Lz

N “4) ‘-4 (‘4 44) 04 i-I N C’)

5
• •— - —-- —~ -

-,-‘-,,,.

~••, —•-‘-—~~~ - 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~ •— -- 

~~~~~~~~~~~~~~~ ‘.~~~~ ‘-‘- ‘~~

- • • • • -.•

~ -

For the criterion of divergence, three of the validity diagonal

values exceeded the respective column and row values, but the remaining

six diagonal terms demonstrated one or more reversals. Even less supportive results were evident for the third criterion , requiring higher

values in the validity diagonals than in the hetero-trait/mono-method

triangles. Only two of the nine values exceeded both of their row ano

column comparison terms.

• An evaluation of the fourth criterion , requiring the same pattern • between variables in the hetero—trait/stono—method section as in the

hetero-trait/hetero-inethod section , suggested that not only were the

• different sources not converging on the same trait but also, in fact,

may have been converging across traits.

The third step in the analysis was based on the analysis of van —

ance model described by Kavanagh et al. (1971). In brief , the model is

a three—way factorial design from which four sources of variance can

be derived to capture the essence of a multitrait-multimethod matrix.

These are (a) leader variance , indicating convergent validity; (b) leader by trait variance, indicating discriminant validity; (c) leader by

source variance, indicating method/source bias; and (d) error (Kavanagh

• • et al., 1971). In addition , the amount of variance accounted for by

each source can be estimated to facilitate relative comparisons anx ng

the variance components; this is not feasible in an analysis restricted

solely to the correlations in a multitrait—multimethod matrix.

The analysis of variance in Table 4 was derived from the correlations in Table 3. The results indicate fairly strong agreement on the

ordering of leaders across the three sources and the three effectiveness

measures (F(22, 88) ~ 9.06, p < .01). However, there was no evidence

for ordering of leaders on different effectiveness measures (discriminant validity) (F(44, 88) = 1.49, N.S.) . There was some limited evidence that source/method halo bias was present (F(44, 88) = 2.00,

p < .05), although it was probably not enough to be of major practical

concern. in short, these results support the interpretations made

using the traditional multitrait-multimethod approach--that is, there

was convergent validity in the data and essentially no evidence for

discriminant validity. But the ANOVA results extended the MT/MM resuits by indicating both the amount of source/method bias and the relatively large amount of error variance involved.

DISCUSS ION

The data indicated a substantial amount of agreement on the overall ordering of leaders when using a variety of techniques to evaluate

leader effectiveness. Further, when leader behavior and group outcome

measures were included , there was relatively little differentiation

between these types of measures, including the influence of source or

relative halo bias (Willingham & Jones, 1958). For example, the participant observers did not seem to distinguish among group relations,

6

~~—~~~—.~~~~~~-• ~~— —• ~~ ---•
ri””’’ • ‘.• ‘

~~‘ ‘ ‘~~~~~~ ‘‘““- ‘~~~~~~~~ r-~ “~~~ “ ~‘ — ‘ ‘.~

.r — “ “_ ‘ ~ • •

• group performance , and leader effectiveness ratings; this suggested

that all their ratings were based on a more limited set of cues than

in the case of group members and leaders who did show more divergence

among these measures.

Table 4

Analysis of Variance of Correlations (from Table 3)

Variance

Source df MS F component

Leader 22 4.26 9.06** .42

Leader x trait 44 0.70 1.49 .08

Leader x source 44 0.94 2.00* .16

Error 88 0.47 .47

• *p < .05.

< .001.

The one leader effectiveness variable that did not seem to be

• related to any of the other variables was peer nomination of leadership

potential. This might be a result of a method problem or a function of

the peer nominations actually tapping an entirely different construct.

Possibly, the respondents closely followed the instructions to nominate

individuals for leadership potentiai; in so doing , they included aspects of behavior and performance that were untapped by the other “hereand—now” measures.

The findings from this study point to the potential for the interchangeability of estimates of leader effectiveness derived from different methods and/or sources; however, the findings also point to a large

amount of error variance. Since most research approaches use criteria

from only one source or method because of convenience or circumstance ,

comparisons of results across studies can be made, but only with appropriate caution . This is because, as indicated in this report, cri—

teria are only moderately comparable, and the degree of convergence

leaves much of the variance unaccounted for in each of the measures.

The limited nature of these data-—in size, measures, and type of sample——

does not permit a definitive statement about the generality of the results; nevertheless, the data point to the need to investigate further

the nature of the relationships between the conmionly used criteria of

leader effectiveness. Furthermore, the data presented here suggest

that different sources or methods may be subject to different types

of bias or insensitivity ; the data also suggest that several different

7

IL11 —
____________ ______ -

~~~ ___

r

methods and sources within the same study may be essential to adequately

tap all aspects of leader performance , as well as to make specific findings interpretable in the context of other studies. As Dunnette (1963a ,

l963b) suggested , the maxim for leader effectiveness measures appears to

be “the more , the better.”

The results of this study have added to a growing body of literature (Mitchell, Larson, & Green , 1977; Rush , Thomas , & Lord , 1977; and

Lord, Binning, & Thomas, 1977) documenting the importance of rater behavior and context or situational cues regarding perceptions of leader

behavior and leadership effectiveness.

ii

8

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ • • • ~~~~~~~~~~ • •

~~~ • -
-, -~~~~~~ ~- 1.- ~- 

~~~~~~~~~~~~~~~~ ~~~- - - ‘ _ ‘

REFERENCES

Campbell , C. F., & Fiske, D. W. Convergent and Discriminant Validation

by the Multitrait-Multimethod Matrix. Psychological Bulletin,

1959, 56, 81—105 .

Downey, R. G. Development of a Measure of Army Leadership Climate: • The Military Leadership Behavior Survey. ARI Research Problem

Review 74—75 , December 1974.

Downey, R. G., Duffy, P. J., & Shiflett, S. Criterion Performance

Measures of Leadership and Unit Effectiveness in Small Combat

Units. ARI Research Memorandum 75-9, July 1975.

Dunnette, M . D. A Note on the Criterion. Journal of Applied Psychology,

1963, 47, 251—254. (a)

Dunnette, H. D. A Modified Model for Test Validation and Selection Research. Journal of Applied Psychology, 1963, 47, 317—323. (b)

Kavanagh, M. J., MacKinney, A. C., & Wolins, L. Issues in Managerial

Performance : Multitrait—Multimethod Analyses of Ratings. Psychological Bulletin 1971, 5, 34—49.

Lawler, E. E. The Multitrait-Multirater Approach to Measuring Managerial Job Performance . Journal of Applied Psychology, 1967, 51,

369—381.

Lord, R. G., Binning , J., Ruch, M . C., & Thomas, J. C. The Effect of

Performance Cues and Actual Behavior on Questionnaire-Base Measures

of Leadership Behavior. Manuscript submitted for publication ,

1977.

Mitchell , T. R., Larson, J. R., & Green, S. G. Leader ~ehavior, Situational Moderators, and Group Performance : An Attributional Analysis. Organization Behavior and Human Performance, 1977, 18,

254—268.

Rush, H. C., Thomas, J. C., & Lord, R. G. Implicit Leadership Theory :

A Potential Threat to the Internal Validity of Leader Behavior

Questionnaires. Organization Behavior and Human Performance,

1977, 20, 93—110.

Stogdill, R. H. Manual for the Leader Behavior Description Questionnaire-—Form XII. Columbus, Ohio: Bureau of Business Research,

Ohio State University , 1963.

Stogdill, R. M. Handbook of Leadership . New York : The Free Press,

1974.

9

- • “•• • • - - . — - -• .• -

~ ~~~~ “—-_-~ —•~ - ~ S• -• ~~~~__~ • • 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ —-----

.

- - -•-- • -- -- - - - •
-

~~ y- -~-

• — • • -•—•-—. •T

Weissenberg, R., & Kavanagh, M. 3. The Independence of Initiating

Structure and Consideration : A Review of the Evidence. Person- ‘

nel Psychology, 1972, 25, 119—130.

Willingham, W. W.,, & Jones, M. B. On the Identification of Halo Through

ANOV. Educational and Psychological Measurement, 1958, 18,

403—407.

Yunker , G. W., & Hunt, 7. G. An Empirical Comparison of the Michigan

Four-Factor and Ohio State LBDQ Leadership Scales. Organizational

Behavior and Human Performance , 1976, 17, 45-65. -

~

F 10 

—~~~ --—. - -——~~~~-- ~~~~~~~~~~~~~~~~~~~ —~~~~~~~~ _

~~~~-• ---a•—-~~—~
I

DI STIIIUTION

AR) 0 c,lsJtusO 145

4 0*50 IM&RA) 2 HOUSACOEC. Ft Ord, ATTh : Library

2 HQQA U)AMI-CSZ)

I 1400* (DAPE PePI 2 USAEEC. Ft Ssni.m,n Harr,son. ATTN: L,lea,y

1 (100* (DAMA AR) 1 USAPACDC, Ft Beniamu, Harrisosi, ATTN: ATCP -HR

I HOOA (OAPF-HRE-P0) 1 USA Cn,nm—ElecI Sth, Ft Montnouth , ATTN~ ATS N-EA

I 1100* (SCRO-t O) I USAEC, Ft Mo,wtwenh, ATTh: AMSEL CT HOP

I HODA (OAM I.DOT.C) 1 USAEC. Ft M~

nmouth , ATTN: AMSEL -PA P

I HO0A (DAPC PMZ Al I USAEC. Ft Monmoutli, ATTN : AMSEL- SI-CS

1 HODA IDACH.PP7 A) 1 USAEC. Ft Minincuth, ATTN: C. Fad Dee Br -

I HODA (DAPE IIRE( I USA Meterisli Sys Anal Agcy . Aberdeen, ATTN AMXSY -P

1 HODA (OAPE-MPO-C) I Edgewood Arsenal. Aberdeen. ATTN: SAREA HI. H

I HODA (DAPE UW) * USA Ord Ctr & Sds , Aberdeen. ATTN: ATSL—TEM-C

1 HODA (DAPE URL) 2 USA Hum Engr Lab. Aberdeen. ATTN~ Library/0~

1 HOOA IOAPE-CPS) 1 USA Combat Arms Ing Sd, Ft Binning. ATTN; Ad Superi.itoq

I HODA (DAFO MFA) 1 USA Infantry Hum Rich Unit. Ft Benninq, ATTN: Chief

1 HODA (DARD-AR S-P) I USA Infint ry Sd, Ft Binning, *11W: STEBC - TE- T

I HODA (OAPC PAS-A) I USASMA. Ft Bliss, ATIN: ATSS LRC

1 HOOA (DUSA OR) 1 USAAir OefSch ,Ft B)in ATTN: ATSA CTO ME

I IlOL)A I DAMO ROR) 1 USA Air Def Sets, Ft Bliss , ATTN: Tech Lili

I lt(M;A (l)A~.G) 1 USA Air Del Sd. Ft Bliss. *11W: FILES

1 HO0A IDA 10.Pl) 1 USA Air Dif Sd. Ft Blits , ATTN: STEBD -P0

1 Clse ’I , Con su lt Div (DA•OTSG) , Adettuhi, MD 1 USA Cmd & General Stf College. Ft Leavenworth. *11W: Lub

1 Mu Asst. Ilsu n Ret . OIM)R&F , 0*1) (E& L S) 1 USA Cmd & General $tf College, Ft Leavenworth , ATIN: ATSW -SE -- L

1 110 IISAHAL flPO Srattlu- , ATTN: ARAGP i

~ 1 USA Cmd & General 5,1 College, Ft Leavenwar it, , ATIN: Eu *dyison

I HO first Army, A T1’N: AFKA•Ol II 1 (ISA Comlui nad Ar ms Crnbt Dcv Act , Ft Leav ’

~

u,w ort h , ATIN: DepCrfr

2 HO Fifth Army, Ft Sans Houston I USA Combined Arms Cmbt Div Act. Ft Leavenworth. ATTN: CCS

1 Dii Army Stf Studi es Ofc , ATTN: OAVCSA I DSP) 1 USA Combined Arms Cntht Div ACt. Ft Leavenworth. ATTh: ATCASA

1 oh Cl,iel of Stt , ~IosIis ~ Ole 1 USA Combined Arms Cmbt 0ev Act , Ft Leavenworth, AIIM: ATCACO—E

I In :sl ’l-R , A1IN: CPS/OCI’ 1 USA Cuvntu ini.d Arms Cmbi 0ev Act . Ft Leavenw orth , AIIM: AICACC-4 I

S liii Aim y Iii ,, l’ci,tsq ui u , ATTN : HSIf (;huuef 1 USAFCOM , Nigtt r Vision Lab , Ft Builvou r , ATTN: AMSEL—NV— SD

1 The Army Liii , Pc’ntaqouu, ATTN: ANIIAI. I LISA Computer Sys Cmd , Ft Belvoir . AITh: Tech Library

1 Ofi , Assi Sect nut the Army (R&D) 1 USAMEROC . Ft Belvoi r , AITN: STSFB--D0

I lunch Support Of C, OJCS I USA Eng 5th, Ft8elvoir , ATIN: Library

USASA, Ar hunu ttot u . ATTN: lARD-I 1 USA Topographic Lab. Ft Belvoir. ATTN: FIL TD—S

I USA Butts Ole, Ot,rham . *11W: Lif e Sciences Dir I USA Topographic Lab , Ft Belvoir , ATIN: STINFO Center

7 (ISA RIEM , N~

dic k , ATTN: SGRD-UE CA 1 USA Topographic Lab, Ft Belvo ir . ATTN: ETI GSL

I (

~;nI ft: It Cheysou n . Al IN ~

;tl it : MI)A I l ISA lu,ts.ll,genr - Ct, A Sdi, ft l4u.iehsu:.a , *11W: CTD MS

I USAIMA , l-t Br.gpl, Al TN: A ISU ClO-OM 1 USA Intelligenc e Ctr & Sth . Ft Huachuca, ATTN: AIS—CID-MS

1 USAIMA , Fr Br,g~

u, , AITN: Marutu at Lib 1 USA Intelli gence Ctr & Sch, Ft Ifuachuca, ATIN: ATSI—IE

1 (iS WAC Ctr & Sd, , Ft McClellan, ATTN : Lii, I liSA Intelligence Cu & 5th, Ft Huachuca, ATTN ATSI- IEX--GS

I IJS WAC Ctr & Sch, Ft McClellan ATTN : Tout I)’, 1 USA Intelligence Ctr & 5th, Ft Huachuca , AITN: AIS)—CTS--OR

1 liSA Ouarte rmast ,-r Sets , Ft Lee, ATTN: AISM-TE I liSA Intelligence Ctr & Sets, Ft Huachuc., ATTN: ATSI—CTD--DT

1 Intelliegnce Materia l 0ev Ole. EWL , Ft Hnta burd 1 USA Intelligence Cu & 5th, Ft Huachuca, ATIN: ATSI-CTO—CS

1 USA SE Siqun al Sch, Ft Gi,ruton , ATTN : ATSO EA 1 USA Intelligence Ct, & Scts , Ft Huachuca. ATTN: DAS/SRD

1 USA Chatutsusu Cu, F, ‘us- t n , F I Uan,itt o ,u . ATTN: ATSC-TF-R0 1 USA Intelligen ce Ct, & Scti , Ft Huechiuca, ATTN ATSI—TEM

I USA1SCIl , F’ Eustis , *11W: Ethic Advisor * USA Intelligence Cl, & ScIs, Ft Huactiuca, ATIN: Library

1 USA War ColIeu~ . Carlisl e Baurack s , *11W: Lu), 1 COR , HO Ft Huachuca , ATIN: Tech Ref Div

7 WRAI R, Nc ii,ut,cyrh uti OW 2 COR , USA Electro nic Pr,y Grd . AIIM: STEFP MT— S

1 DLI , SOA . Muu,,tr,e-1 1 HO, TCATA. AITN: Teds Library

1 lISA Concept An al Ailcy. Rsnthesda, ATIN: MOCA MR I 140, TCATA, ATTN: ATCAT’OP’O. Ft Hood

I USA Conce pt A,,a) Agey . Bethesda, AITN: MOCA-JF 1 USA Recruiting Cmd, Ft Sheridan. ATTN: USARCPM-P • - 

I USA Arctic Test C*r , API) Scattl~, ATTN: STEAC-PL-Ml I Senior Army Mv.. USAFAGOD/TAC. Elgin AF Aux Flit No. 0

1 USA Arctic Test Ctr , APO Seattle, ATTN: AMSTE.PL-1S I HO. USARPAC. DCSPER. APO SF 95555. ATIN: GPPE-SE

I USA Armament CmuI , Retltton, Arsenal . ATTN: ATSK’TEM 1 Stirn,on Lib. Academy of Hs th Sciences, Ft Sam Houston

* USA Armament Crnd. Rock liland, AIIM: AMSAR-TOC 1 Marine Carps Inst., ATTN: 0e n-MCI

• I FAA.NAFEC , Atlantic City, AIIM: Library I HO, USMC. Commandant. *11W: Code MTMI

I FAA NAFEC. Atlantic City. ATTN: Human Engr Br 1 HO. USMC. Commandant. AIIM: Cods MPI-20-a

1 FAA A.ron utical Ct,. Oklahoma City, ATIN: AAC-44D 2 USCG Academy. New London, AIIM: Admission

2 USA FM Arty 5th, Ft Sill, AIIM: Library 2 USCG Academy, Wise London, AIIM: Library

I USA Armor Sch, Ft Knox, AIIM: Library I USCG Training C,,, MV . *11W: CO

~~~~~~ 

~

‘ (ISA Armor 5th, Ft Knox , AIIM: *158-0)-F 1 USCG Training Ctr. NV , *11W: Iuliac Sec Ofc

-: I USA Arm,,, Sets , ft F”.,~. AT TN A1’SB 01 IP * USCG, Ptychal Ret B., DC. ATTN GP 1/62

I USA Armor Sth, It Knor, ATTN ATSB•C0-AD 1 HO Mid-Range Br, MC Oat. Ou.ntico, *11W: P50 Div

11

• 

-
~~~~~~~~~~ • • •~~~~~~ -~~~ —

1 US Marine Corps Luaison Ofc, AMC, Alexandria, ATTN - AMCGS -r 1 Dcl & Civil Inst oi Enviro Medicine. Canada

* USATRADOC, Ft Monro, , AIIM: AIRO’-ED 1 AIR CRESS. K.neington, AIIM: Into Sys Br

6 USATRAOOC, Ft Monro, AIIM: ATPR AD I Miluta.rpsykalogidi Tjeneste. Copenhagen

I USATRAOOC. Ft Monroe, ATT M: AIIS--EA 1 Military Attach., French Embassy. AIIM: Doc Sac

1 USA Forces Cmd, Ft McPherson, AIIM; Library 1 Medecin Ch.t, C.E.R.P A. --Arsenal, Touloi,/Naval France

2 USA Ayia~

ion Teat 84, F) Runcker, AIIM: STEBO- P0 1 Prin Scientific Off , AppI Hum Engr Rsth Div. Ministry

1 USA Agey ton Aviation Safety, Ft Riacker . AIIM: Library of Defense. New Delhi

1 USA Agcy to, Aviation Safety, Ft Rocker, AITN; Educ Anvitor I Pars Rsch Ofc Library, AKA, Israel Defense Fo.ets

I USA Aviation Sets. Ft Rucker. AIIM: P0 Drawer 0 1 Mini,t,ris van D.fansie. 000P/KL At d Sociaal

1 HOUSA Aviation $ys Cmd, St Louis. AIIM; AMSAV—ZDR Psychologiache Zaken, 11w Hague. Nethert nds

2 USA Aviation Sy~ Ten t Ac,., F,dward’e AFB. AIIM: SAVIE - -T

1 USA Air Del Sets, Ft NIh’,, AIIM; ATSA IFM

I liSA AU Miiluility Rstli & Dine Lab, Mofteft Flit, ATIN: SAVOL -AS

1 -USA Aviation Sets, Re, Trig Mqt, Ft Rocker , AIIM: AIST -I--RIM

1 USA Aviation $cti, CO. Ft Rocker, AIIM; ATSI—D’-A

1 HO, DARCOM. Alexandria, AIIM: AM*CD -IL

I HO, DARCOM. Alexandria, AIIM: CDR

1 US Military Academy, Watt P oint, AIIM: Serials Unit

1 US Military Academy, We,t Point, AIIM: Ofc of Mill Ldnslu t

1 US Milftaiy Academy , West Point, AIIM: MAOR

1 USA Standaiuii,ation Op. UK , FPO NV , AIIM: MASS -GC

1 Dec of Naval Reds. Arlington, AIIN: Code 452

3 Ole of Maval Rtch, Arlington, AIIM: Cods 468

1 Otc of Naval Rich, Arlington, AIIM; Cods 450

1 Ofc of Naval Rich, Arlington, AIIM: Code 441

I Naval Aerrsspc Med Ret L.h, Pensacola, AIIM: Acorst Sets Div

1 Naval Acrospe Med Ret Lab , Pensacola, AIIM: Code L51

1 Naval A*ruuspc Med Ret Lab, Pansacola, AIIM: Code LS

1 Chiuil of NavPu,rs, AIIM: Pert- OR

I NAVAI RSTA, Norfolk , ATTN ; Safely Cli

1 Mar Oceanographic, DC, AIIM; Code 6261, Charts & Tech

1 Center of Naval Anal , ATIN; Doe Ctr

L I NavAirSysCom, AIIM: A lR --

~~1X

1 Na,. BuMed, AIIM: 713

I NevHellcoptarSubScius 2, FPO SF 95601

I AFHRL (Fl) Williams AFB

1 AFHRL )TI) Lowey AFB

1 AFHRL (AS) WPAFB, OH

2 AFHRL (OOJZ) Brooks AFB

1 AFHRL (OOJN)LackI.nd AFB

1 HOUSAF (INVSO)

f HOUSAF (OPXXAt

I AFVIG IRDI RandoIOIs AFB

3 AMRL (HE) WPAFB , OH

2 AF Inst of lunch , WPAF B, OH. AIIM: ENE/6L

I *1’C (XPIOI Randoft,h AFO

1 (ISAF AeruiMund Liii, Broruks AFB tSUL - 4), AIIM: DCC SEC

1 AFOSR (NI). Arlingtous

• 1 AF Log Cmd, McCluiIl,in AFB . AIIM: ALC/DPCRB

• I Air Fort, Academy, CO. ATIN; Dept of S.f Sen

5 NavPert & Dee Cu. San Diego

2 Navy Med Nvur.’.~

svth iat,ic Rids Unit , San Diego

1 N.,. Electronic Lab, San Diego. AIIM: Ret Lab

1 Mae IrngCen, San Diego. AIIM: Code 9000.-Lit,

I NavPostGreScIi, Munli’ui’y, AIIM: Code 56*.

I NavP,ustGrvSch , Montcruiy AIIM: Cod. 2124

I N..vlrngEquipCtr, Orlando, AIIM Tech Lib

I US Dept of Labor. DC, AIIM: Manpower Adm,n

I US Dept of Justice, DC, AIIM: Drug Enforc. Admin

I Nat Bur of Standards, DC, AIIM; Computer Info Section

1 Nat Clearing House for MH- Info, Rockville

I Denver Federal Ctr, LSIIIWOOd, A1’TN; BLM

*2 Detente Documentation Center

4 Dir Psych. Army Hq, Ruis.fl Oft,, Canberra

1 Scientific Adv,r, Nil Bd, Army (4g. Russell Ofcs, Canberra

I Nil and Air Attache, Auitrian Imttesey

I Centie de Recherche Des Facteurs. Humaine di, Ia Defense

Natinnad , Brsistrhs

2 Canadian Joi,,t Staff Washington

I C/Air Staff , Royal Canadian AF , AIIM- Pert SM Anal Or

3 Chief , Canadian Del Rich Staff . AIIM: C/C~ DS(W)

I Buntis h Del Staff , British Emliatsy, Washington

12 

- -— .t_ ,,—-

~ —. ——a ____ — -‘------ — --