Research Report 1418

Recommendations for Improved

Performance Appraisal in the Federal Sector

Alma G. Steinberg and William P. Burke

Leadership and Management Technical Area

Manpower and Personnel Research Laboratory

rJUN 2 18

U. S. Army

Research Institute for the Behavioral and Social Sciences

January 1986

A pr o v o (II ,r ptitlr EIt ,1eas ii (I If iI i c u Ii f ) I . ,e

86 6 27 002
" 1 "w"I" -'''V* . . W _ Y . i . u . 4 v T • W . - -. . - P, - .~ - .- ,- : , - ;. I- . ;.. ". . ". .-" aJiU. S. ARMY RESEARCH INSTITUTE

FOR THE BEHAVIORAL AND SOCIAL SCIENCES

* A Field Operating Agency under the Jurisdiction of the

Deputy Chief of Staff for Personnel

WM. DARRYL HENDERSON

EDGAR M. JOHNSON COL, IN

Technical Director Comma~nding

*Technical review by

* Michael G. Rumsey r

Paul Van Rijn

Doris C. Parker, Office of Labor and Employee Relations (ODCSPER)

Conrad M. Lacy, Planning and Evaluation office (ODCSPER)

Tl

• I ,

NOTICEST El

DISTRIbUTION: Primary distribution of this report has been made by ARI. Please address correspondence concerning distribution of reports to: U.S. Army Research Institute for the Behavioral

and Social Sciences, ATTN: PERl-POT, 5001 Eisenhower Ave- Alexandria, Virginia 22333-5600.

FINAL DISPOSITION: This report may be destroyed when it is no longer needed. Please do not

return it to the U.S. Army Research Institute for the Behavioral and Social Sciences.

. . .~~> .

TcNOTE: Direo fidnsiChsrpr r o ob osre , noffilDrmntng eAfm

poitioules soevi by nae d ote uhrze ounn
SECURITY CL Asstric^TIo. or imi e^G (-h_ t)_~ p

REPOR DOCMENTTIONPAGEREAD INSTRUCTIONS REPOR DOCMENTTIONPAGEIIEFORE: COMPLETING F-ORM

I. REPORT NUMBER Z. GOVT ACCESSION No. 3. PECIPICN7*5 CATALOG NUMBER

4. TI TLEC (and S.betl.) 5. TVPE_ or REPORT A PERIOD COVEREDRECO MiNENDATION 'S !OR IMPROVED PERFO PMANc r_ December 1982-December 1985]

7. AUTHOqR(.) 8. CONTRACT OR GRANT NUMBER(.)

Alma G. Steinberg and willian P. Burke

9. PERFORMING ORGANIZATION NAME AND ADDRESS 10. PROGRAM ELEMENT. PROJECT. TASK

U.S. Army Research Institute for the Behavioral AE OKUI UBR

and Social Sciences 2Q.26374 3A794

5001 Eisenhower Avenue, Alexandria, VA 22333-5600 451 7700

It. CONTROLLING OFFICE NAME AND ADDRESS 12. REPORT DATE

U.S. Army Research Institute for the Behavioral January 1986

and Social Sciences 13. NUMBER ofPAGES

5001 Eisenhower Avenue, Alexandria, VA 22333-5600 45 14. MONITORING AGENCY N4AME II AOORESS(II dllf.1.,.I-o CoC"tolfnj Ofice) IS. SECURITY CLASS. (of thlo report)

Unclassified

IS.. DECL ASSI FIC ATION/ DOWNGRADING

SCHEDULE

16. DISTRIBUTION STATEMENT (of tht. R.po-t)

Approved for public release; distribution unlimited

17. DISTRIBUTION STATEMENT (of the ab.I,.ct entered In Block 20, if difV.,-,,I Soo R.p.,t)

IS. SUPPLEMENTARY NOTES

19. K EY WORDS (C..tInuo ,n rov... ide If neC-ea.,y and identify by block -. nb.,)

Performance appraisal

P'erformance appraisal systems

Manlagement by% objectives

Officer Evaluation Report (OEIR)

20. A&STWACT CCC Jf of -- I -4 #-ItIf hi lck -b.)

Federal agencies covered by the Civil :;ervice Reform Act: of 11978 are re- quired to have performance appraisal 5vts;that are to be used as a basis

for personnel actions such as training, rew~irding, promoting, andI removing( em- ployees. As such it is essential that the,( :.erformance appraisal1 syst-ems- be

viewed] as e ffectijve by both supervisjois an~l subordinate!-,. In order to ident ify

the features of performance appraisal ;'t.wthat are iUart. icul1ai I%, effect ive

,ind wh icl al1so meet the legal re'iu ireme nt ; 'or I esetde ra 1aenlC; a

DO~ 1473 EDITION OF I NOV GS IS OIF$O.E TC 1C!A! I FI I")______

StI RITV CL_ A%-SjC ATfM of THIS PAGIC (WI'-n 0).10 FnI.,d)
UNCILASSIFIED

,.t CUS1T ' CL ASSIFICATION Of T.fS i'A6(( t) }.re .l.

ARI Research Report 1418

20. (Continued)

review of the performance appraisal literature was conducted and federalsector subject-matter experts were interviewed. The recommendations derived

from the literature review and interviews were organized around the following nine themes: legal requirements, user perspective, the role of top

management, appraisal as a management tool, linkage to organizational goals,

adaptation to organizational needs, system perpetuation, training, and ease

of use.

1-

~I A ... ] I I i:, ,;

ii f( j)i,'r (I. *t10-- A..) ,0 .. ,..r . ., o~o.. ,.,
Research Report 14 18

Recommendations f or Improved

Performance Appraisal in the Federal Sector

Alma G. Steinberg and William P. Burke

Leadership and Management Technical Area

William W. Haythorn, Acting Chief

Manpower and Personnel Research Laboratory

Newell K. Eaton, Acting Director

U.S. ARMY RESEARCH INSTITUTE FOR THE BEHAVIORAL AND S~OCIAL SCIENCES

5001 Eisenhower Avenue, Alexandma, Virginia 22333--K-600

Office. Deputy Chief of Staff for Personnel

Department of the Army

January 1986

Army Project Number Education and Training

20263743A 794

Approve'd for Cputlic ru (ai' ISI~bution tinlirmtod,
A RI Research ReI)or ts and T echnical RepiIs are lntendc(1 to( sponsors of

R&D tasks and for other research and military agencies. Any findings ready

for implementation at the time of publication are presented in the last part

of the Brief. Upon completion of a major phase of the task. formal recoinmendations for official action normally are conveyed to appropriate military

agencies by briefing or Disposition Form.
.-

F09EWORD

The purpose of this research was to identify effective features of performance appraisal systems that might be considered for inclusion in the Department of the Army's performance appraisal system for civilian personnel. Since

this system is required by law to meet the same design and functional requirements as other federal performance appraisal systems covered by the Civil Service Reform Act of 1978, the approach taken was to both review the literature

on performance appraisal and to interview subject-matter experts of other federa! agencies in order to learn about particularly effective features of their

systems. This report is likely to be of interest to both civilian and military

managers, supervisors, and personnelists, within the Department of the Army and

in other federal agencies, as it is concerned with performance appraisal, performance management, and accomplishment of organizational goals.

EDGAR M. JOHNS 6 N

Technical Director

Vi I

..........
* • ... . . . . . . . . . . . . . .

* RECOMMENDATIONS FOR IMPROVED PERFORMANCE APPRAISAL IN THE FEDERAL SECTOR

EXECUTIVE SUMMARY

Requirement:

To identify those features of performance appraisal systems that are particularly efiective and that also meet the legal requirements for federal agencies covered by the Civil Service Reform Act of 1978.

Procedure:

Two approaches were used. The first was a review of the performance appraisal literature, with special emphasis on performance appraisal systems, or

features of systems, which are consistent with the Civil Service Reform Act requirements. The literature included standard, current reference works in the

field of performance appraisal, relevant original sources cited in these references, and all relevant articles from 10 personnel/management journals from

1982 through the present.

Since very little literature is available on current federal performance

appraisal systems, a second approach was used to obtain specific information

* about effective features of these systems. More than 100 interviews were conducted with federal-sector subject-matter experts who had roles in developing,

administering, monitoring, and/or evaluating their agencies' systems.

Findings:

Nine prescriptive themes for more effective federal performance appraisal

* systems are discussed. The themes, which emphasized the importance of using a

systems approach in making performance appraisal work as Intended, were as fol-

* lows: (1) legal requirements must be met, (2) performance appraisal needs to

be viewed as important and worthwhile by users, (3) top management backing is

essential, (4) the performance appraisal process should be used as a management

too!, (5) the performance appraisal process should be linked to organizational

goals, (6) the performance appraisal system should be adapted to organizational

needs, (7) system perpetuation should be planned, (8) training is essential,

and (9) ease of use is important.

* Utilization of Findings:

The findings of this study should be useful to management, supervisory, and

* personnel audiences in federal agencies covered by the Civil Service Reform Act

of 1978. They will enabi'o these audiences to understand their role in the federal performance appraisal system, how the system can facilitate the accomplish-

* ment of organizational go~is, and what is needed to make It work.

* * .4
RECOII1MENDATIONS FOR IMPROVED PERFORMIANCE APPRAISAL IN THE FEDERAL SECTOR

CONTENTS

Page

LINTRODUCTION. ............................... 1

Literature Review. .. ..........................

Interviews with Subject-matter Experts. ................ 2

PLESSONS LEARNED .. ............................. 3

Legal Requirements Must be Met. .................... 4

Title VII of the Civil Rights Act of 1964 ............. 4

Civil Service Reform Act of 1978 .. ................ 5

Civil Service Requirement Spouse Equity Act of 1984 .. ....... 5

Differing Legal Requirements for Civilian and

Military Personnel .. ...................... 6

Performance Appraisal Needs to be Viewed as Important and

Worthwhile by Users .. ........................ 10

Top Management Backing is Essential .. ................. 13

The Performance Appraisal Process Should be Used

as a Management Tool. ........................ 13

The Performance Appraisal Process Should be 'Linked

to Organizational Goals .. ...................... 16

The Performance Appraisal System Should be Adapted to

Organizational Needs. ........................ 17

System Perpetuation Should Be Planned .. ................ 18

Training is Essential .. ........................ 19

Ease of Use is Important. ....................... 22

Generic elements and standards. .................. 22

Rating scale formats. ....................... 23

CONCLUSIONS. ............................... 28

REFERENCES......................................31

ix
.. ~

LIST OF APPENDICES

A. Example of Agency Statement of Goals and Support for MBO ....... A-I

B. Example of Statement of Top Management Support for Training . . . B-1

LIST OF TABLES

1. A comparison of some components of appraisal systems under

CSRA and the Army Officer Evaluation Report (OER) ... ....... 8

2. Competencies and attributes rated on Part IV of the OER .. ..... 9

3. Lack of enthusiasm for performance appraisal .... .......... .11

4. Performance appraisal systems seen as failures ... .......... .12

5. Steps of the performance appraisal process .... ........... .15

6. Performance appraisal training program topics .... .......... .21

x

I '
L

RECOMMENDATIONS FOR IMPROVED PERFORMANCE APPRAISAL

IN THE FEDERAL SECTOR

INTRODUCTION

The Civil Service Reform Act (CSRA) of 1978 directed specified Federal

agencies, including the Department of the Army, to establish performance

appraisal (PA) systems that would be used as a basis for training, rewarding,

promoting, and removing employees as well as other personnel actions. In

compliance with specific provisions of CSRA, the Department of the Army established the required PA systems for its various categories of civilian

personnel (Army Regulation No. 690-900, 690-500, and 690-400).

Informal feedback from within the Department of the Army has revealed a

lack of enthusiasm for the current PA systems. Army civilian PA is seen by

many as overly complex, too time-consuming, and of uncertain value.

Consequently, the Director of Civilian Personnel for the Army (Office of the

Deputy Chief of Staff for Personnel), requested that the U.S. Army Research

Institute (ARI) conduct research to identify the parameters of an improved PA

system for the Army.

The research conducted by ARI in response to this request was designed to

identify those features of PA systems which have been particularly effective

and which also meet the legal requirements ior Federal agencies covered by

CSRA. Three approaches were used to accomplish this: a review of the PA

literature, a survey to identify successful features of current Federal

agency PA systems (see Steinberg & Burke, in press), and interviews with

subject-matter experts.

This report is divided into two parts. The first is a method section

that describes the literature review and interviews with subject-matter

experts and the second is a synthesis of the findings organized around nine

themes.

METHOD

This section describes the two sources of information upon which the

themes and conclusions presented in later sections are based. The two

sources are selections from the PA literature and interviews with

subject-matter experts.

Literature Review

The literature in the field of PA currently is vast, and is growing at a rapid rate. Literally thousands of articles have been published on the topic

within the last 30 years (Henderson, 1984). Consequently, an attempt was

made to limit the review to literature emphasizing systems, or features of

systems, consistent with CSRA requirements.

-14
The literature reviewed includes a mix of empirical studies and expert

opinion. Although the latter may be based on untested hypotheses, it is

included because it representis the recommendations of experts in the field.

The review began with standard, current reference works in the field of

PA (e.g., Baird, Beatty, & Schneier, 1982; Bernardin & Beatty, 1984;

Conference Board, 1979; DeVries, Morrison, Shullman, & Gerlach, 1980; Landy &

Farr, 1980; Latham & Wexley, 1981; McCall & DeVries, 1977). Then, original

sources cited in these references were reviewed. Their selection was based

on the relevance of their cited titles and content or their repeated mention

in the literature. These sources, in turn, often suggested other pertinent

references. In addition, recent (1982 through the present) volumes of

frequently cited journals were searched for relevant articles. These

journals Included the following:

o Academy of Management Journal

o Academy of Management Proceedings

o Academy of Management Review

o Journal of Applied Psychology

o Management

o Management Review

o Personnel Journal

o Personnel Psychology

o Public Personnel Management

o Training and Development Journal

Interviews with Subject-Matter Experts

Very little literature is available on current Federal PA systems.

Therefore, a survey of Federal agency subject-matter experts was conducted in

order to learn about these systems. The survey procedures are described

elsewhere (Steinberg & Burke, in press) but the results are considered here

in conjunction with the rest of the literature reviewed. Its relevance in

this section is that in the process of developing and administering the

survey, and also subsequent to its administration, both formal and informal

interviews were conducted with more than 100 subject-matter experts. These

interviews are considered another source of information for the purposes of

the present report.

The surveys were mailed to the members of the Committee on Performance

Management of the Office of Personnel Management (OP') Interagency Advisory

Group (IAG). The members of that committee, each representing a different

Federal agency, play a major role with respect to their agencies' PA systems.

2
They have responsibilities for developing, administrating, monitoring, and/or

evaluating their agencies' systems. Their job titles include Program

Manager, Personnel Management Specialist, Agency Coordinator, Director of the

Office of Performance Evaluation, Chief of Personnel Programs and Policy, and

Performance Systems Officer.

The interviews were conducted in the following contexts:

o Federal sector personnel managers Intimately knowledgeable about PA

were consulted for background information utilized in the design of

the survey.

o Additional Federal sector subject-matter experts participated in the

pretest of the survey. During these sessions, participants related

their ideas and experiences.

o Telephone contact was made with each of the 88 lAG committee members

to enlist participation In the survey. Many used this opportunity

to air their concerns and recommendations regarding PA.

o Followup interviews were conducted with a number of survey

respondents to obtain more detailed information about their

recommendations and also with other subject-matter experLs

recommended by the survey respondents.

Note that the interviews with subject-matter experts were not used to

evaluate researc, ,ypotheses. Rather, they were used to elicit thoughts and

ideas based on experience in the PA arena.

LESSONS LEARNED

Nine prescriptive themes emerged from the integration of the literature

review and the interviews. These themes summarize the parameters of an

effective PA system for Federal agencies under CSRA. These themes are as

follows:

A PA system appears to work when:

o legal requirements are met.

o PA is viewed as important and worthwhile by the users.

o top management backs it.

o appraisal is used as a management tool.

o appraisal is linked to organizational goals.

o it is adapted to organizational needs.

3

. . . . . ,. . . .. . -.. . ... . .. ,_- .-. . - . - ..-. .- -. , .", '',- - . - - - -
o it includes mechanisms for system perpetuation.

o appropriate training is provided.

o It is easy to use.

The section below contains a discussion of each of these themes in turn. .,

Legal Requirements Must be Met

The first lesson learned from both the literature review and the

interviews is that a PA system must be designed with legal requirements in

mind. Appraisal systems are being challenged in the courts with increasing

frequency. This has led to an increased awareness of, and attempts to comply

with, the legal constraints within which an appraisal system should be

designed (Edwards, 1983; Schneler & Beatty, 1979). This awareness of legal

constraints is heightened by the fact that the cost to agencies found in

violation of the law can easily run into millions of dollars (Latham &

Wexley, 1981). Three major pieces of legislation affecting the design of PA

systems in the federal sector are discussed below.

Title VII of the Civil Rights Act of 1964. Title VII of the Civil Rights

Act of 1964 (P.L. 88-352) prohibits discrimination in employment decisions.

The result of court actions initiated under the provisions of this act have

made it clear that PA is among the employment practices covered by it

(Lubben, Thompson, & Klasson, 1980). Further, the Uniform Guidelines on

Employee Selection Procedures (1978) provide that if an employment practice

(e.g., PA) is shown to have adverse impact, the employer must demonstrate

that the employment practice has been validated in accordance with the

Uniform Guidelines. This validation must begin with a formal job analysis of

the position in question (Lubben et al., 1980). "There should be a review of

job information to determine measures of work behaviors or performance that

are relevant to the extent that they represent critical or important job

duties, work behaviors or work outcomes as developed from the review of job

information. In view of the possibility of bias in subjective evaluations,

supervisory rating techniques should be carefully developed; and the ratings

should be examined for evidence of racial, ethnic or sex bias" (Uniform

Guidelines on Employee Selection Procedures, 1978, p. 38300).

Prescriptions for PA systems typically include the recommendation

for a job analysis not only as sound personnel practice, but also as a

precautionary measure in case the system is challenged (Bernardin & Beatty,

1984; Heneman, Schwab, Fossum, & Dyer, 1983; Lubben et al., 1980). (A PA

system lacking certain critical features, which include a job analysis, has

been equaled to "a cocked gun pointed at the employer" Landy, 1985, p. 171.)

Indeed, a court challenge may occur even in the absence of adverse impact.

As Kleiman and Durham in their review of 23 Title VIl court cases have

concluded: "the court has shown interest in assessing performance appraisal

systems regardless of their adverse impact" (1981, p. 103).

4
Civil Service Reform Act of 1978. The Civil Service Reform Act of 1978

(P.L. 95-454) and the OPM regulations which implement it (Office of Personnel

Management, 1979) call for PA systems which include the following features:

o At the beginning of each appraisal period, a performance plan should

be written. This plan will contain:

- the critical elements of the employee's job (i.e., duties and

responsibilities of the job for which performance below the

minimum standard requires remedial action and denial of a

within-grade increase), and

- the standards by which performance on these critical elements

will be evaluated. The performance standards must be objective

criteria related to the duties and responsibilities of the job.

They may include, but are not limited to, factors such as

quantity, quality, and timeliness.

o Employees are encouraged to participate in the establishment of the

performance standards.

o No preestablished rating distribution is permitted, such as a forced

distribution on a bell curve. Appraisal must be based on actual

performance on the job and preestablished objective standards for

performance.

Civil Service Retirement Spouse Equity Act of 1984. Title 11 of the

Civil Service Retirement Spouse Equity Act of 1984 (P.L. 98-615) amends

portions of Title V of the United States Code, dealing with PA systems. (The

Title II portion of the act is based largely on S.958, proposed Civil Service

Amendments of 1984, and does not deal with retirement or spouse equity.) This

legislation and the regulations that implement it mandated the Performance

Management and Recognition System (PMRS) to replace the Merit Pay system for

Federal supervisors and managers, and modified and/or added specific features

to the provisions of CSRA. These are as follows (Office of Personnel

Management, 1985):

o Training and information is now required for PMRS employees and

their supervisors on the purpose of the Perfor-mance and Management

Recognition system and how it works.

o Uniform five-tiered summary rating levels for PMRS employees are

specified. They are "Fully Successful," with two levels above it,

such as "Exceeds Fully Successful- and -Outstanding," and two levels

below it, such as "Minimally Successful" and -Unacceptable".

o Performance of PMRS employees on assigned details of 120 or more

days within or outside the agency should be taken into account in

determining ratings. If details are within the agency, elements and

standards must be written for it.

5

--:-.-.~~~ . . - i, - , ~~ ...- ... .... .- .. .. . . . -. .-.- - - -z .. - . . .- .: ,-, -. .. ,-
o PMRS appraisal periods must end not earlier than June 30, and not

later than September 30 of each year.

o Joint participation between PKRS employees and their supervisors in

developing performance plans is required. (For non-PMRS employees,

it remains "encouraged.")

o PMRS performance plans must be reviewed at a higher level in the

organization and higher-level review of appraisals prior to

providing ratings to PMRS employees is required.

o A progress review must be held at least once during the appraisal

period for PMRS employees.

o One or more 6-member performance standards review boards should be

established for each agency to review representative performance

plans for quality of the plans and difficulty of the performance

standards and to report and make recommendations to the head of the

agency or designee. One half of the board must be composed of PMRS

employees.

Although the above items refer specifically to PMRS employees, the

regulations emphasize performance management and the linkage of PA to

the accomplishment of organizational goals for non-PMRS employees as well.

The term "performance management" is introduced and replaces the term

"performance appraisal" in portions of the regulations. Although

"performance management" is not defined separately, the term "performance

management plan" is. That is the "agency's methods which integrate

performance, pay, and incentive systems with its basic management functions

for the purpose of improving individual and organizational effectiveness in

the accomplishment of agency mission and goals" (Office of Personnel

Management, 1985). Thus, the performance management system is an integrated

management system; and the performance management plan includes the PA plan.

The emphasis on the linkage of PA to the accomplishment of organizational

goals also can be clearly seen in the new Subpart D which has been added to

the OPM regulations which implement CSRA (Office of Personnel Management,

1985). The stated purpose of this subpart entitled "Performance Appraisal

for the Performance Management and Recognition System" is to "ensure that

performance appraisal systems for PMRS employees are used as a tool for

executing basic management and supervisory responsibilities by:

(a) communicating and clarifying agency goals and objectives; (b) identifying

individual accountability for the accomplishment of organizational goals and

objectives; (c) evaluating and improving individual and organizational

accomplishments" (p. 11791). The subpart requires that accomplishment of

organizational objectives be included in PMRS performance plans by

"incorporating objectives, goals, program plans, work plans, or other similar

means that account for program results" (p. 11792).

Differing Legal Requirements for Civilian and Military Personnel.

Although it has been suggested by some that the Army Officer Evaluation

Report (OER) might stand as a model for a civilian PA system, a civilian

system cannot be adapted readily from the military due to the differing

6
requirements of legislation that cover each of these groups. Title VII of

the Civil Rights Act of 1964 has been extended by the Equal Employment

Opportunity Act (1972) to govern both public and private sectors (Bernardin

& Beatty, 1984). However, by ruling of the Department of Defense General

Council, Title VII does not cover military personnel (Eitelberg, Laurence,

Waters, & Perelman, 1984). Also, members of the uniformed services are not

covered by CSRA, since they are not considered "employed in or under an

agency" (Performance Rating, 1977, p. 549). Instead, active duty military

personnel are governed by Title X of the United States Code (Eitelberg et

al., 1984).

The PA systems for civilians and military personnel within the Department

of the Army reflect these different legal philosophies (see Table 1). The

civilian system emphasizes the evaluation of the employees' behavior against

the specific requirements of the job, whereas the military system evaluation

incorporates attributes and competencies of the individual and comparison of

the individual with others of the same rank (Department of the Army, 1985;

Pelissero, 1984). Thus, for example, a GS-11 engineer is rated against the

job requirements and performance standards for a particular job in a given

department at a given installation, whereas an Army Captain engineer doing

the exact same job is evaluated not only on performance of duty requirements

(no prior standards provided), but also on (a) potential compared to all

other Army Captains, and (b) competencies and attributes such as

adaptability, integrity, moral courage (see Table 2). In the the military

system, no guidance is provided in the regulations on what is meant by these

latter dimensions (Department of the Army, 1985). Supervisors provide their

own Interpretations of -moral standards." In the civilian system under CSRA,

in order to evaluate competencies and attributes, these must be stated in

behavioral terms and shown to be specifically related to the requirements of

the job. Determination of job-relatedness, however, must be based on a job

analysis for the particular job in question. It is highly unlikely that one

overall list of competencies could be used for all jobs (Pelissero, 1984).

7

|1

" .. . ... . . . - - • . * . . ..• " -' " ". " . -. .- ' . - - " " " " -. ' ' ' .' L - -" " " . " " .".'
Table 1

A Comparison of Some Components of Appraisal Systems under CSRA and

the Army Officer Evaluation Report (OER)

Appraisal System Component Performance Appraisal Under:

CSRA OER

Critical elements that are job- Required Not Requireda

specific and objectively defined

at the beginning of the rating

period

Written, objective, job-related Required Not Required

standards, defined at the beginning of the rating period,

against which each rating is

made.

Preestablished rating distribution Not Permitted Requiredb

Non-job-specific elements Not Permitted Requiredc

a An element is critical if it is "a component of an employee's job that is

of sufficient importance that performance below the minimum standard

established by management requires remedial action and denial of a

within-grade increase.. .Such action may be taken without regard to other

components of the job" (Office of Personnel Management, 1979, p. 45590).

Under the OER system, although job-specific elements may be defined at

the beginning of the rating period, determination of criticality is not

required.

b The senior rater is required to assume a bell-shaped normal distribution

for officer potential and then compare the rated officer's potential with

all other officers of the same grade rated by the senior rater

(Department of the Army, 1985).

C Regardless of grade, position, branch, and specialty, all Army officers

are rated on the competencies and attributes presented in Table 2.

8

........ .......................................... ....
Table 2

Competencies and Attributes Rated on Part IV of the OER

Professional Competence

o Possess capacity to acquire knowledge/grasp concepts

o Demonstrates appropriate knowledge and expertise in assigned tasks

o Maintains appropriate level of physical fitness

o Performs under physical and mental stress

o Encourages candor and frankness in subordinates

o Clear and concise in written communication

o Displays sound judgment

o Seeks self-improvement

o Is adaptable to changing situations

o Sets and enforces high standards

o Possesses military bearing and appearance

o Supports EO/EEO

o Clear and concise in oral communication

Professional Ethics

o Dedication

o Responsibility

o Loyalty

o Discipline

o Integrity

o Moral Courage

o Selflessness

o Moral standards

9
Performance Appraisal Needs to be Viewed as Important and Worthwhile by Users

In order for the PA process to be used with more than minimal, begrudging

compliance, it needs to be viewed as important and worthwhile by the users.

Yet the very opposite exists in many organizations. The PA process is

frequently viewed with great pessimism. It is seen as an unpleasant burden

and its usefulness is questioned (see Table 3). Past PA systems often are

seen as failures and little optimism for the success of future systems exists

(see Table 4). These negative views toward PA may, however, be partly

responsible for the failure of these systems. Lack of user support for PA

undermines the system (Abbott & Schuster, 1984; Barrett, 1966; Hall, 1983;

Kirkpatrick, 1984).

An example of the impact of lack of user enthusiasm on the implementation

of a PA system occurred In one Federal agency (Gaertner, Gaertner, & Akinnus,

1984; for other examples see Conference Board, 1979, and Hall, 1983). Top

administrative officers were highly committed to the use of PA, lobbied

extensively for the passage of CSRA, and upon passage of the act, volunteered

to put Merit Pay in place a year ahead of the mandated implementation date.

However, the agency's operative level personnel resisted and the major

program offices successfully argued that they be excused from early

implementation of the act. Gaertner et al. (1984) attributed the resistance

by operating-level personnel to the: (a) lack of confidence in the chain of

command, (b) distrust of agency management, (c) political atmosphere within

the agency, and (e) fear that the innovations would reduce employee autonomy.

Unions also may resist PA systems and seek to ensure that their

interests are protected. They traditionally view seniority (i.e., length

of employment) rather than PA, as the basis on which benefits should be

allocated and administrative decisions about promotion, layoffs, and wage

increases should be made (Gordon & Johnson, 1982; Ingster, 1984). The

Federal Labor Relations Authority (FLRA) has had to resolve some performance

appraisal system disputes between unions and Federal agencies regarding the

rights of each. (Examples of decisions on which aspects of Federal agency PA

systems are negotiable between unions and management and which are outside

the duty to bargain because they are a violation of management right to

direct employees and assign work can be found in United States Code Service,

1985, pp. 210, 237.)

The subject-matter experts and the PA literature both strongly suggested

that, when revising a PA system, it is important to involve the unions early.

There should be a continuous flow of information to the union about the

appraisal plan (Ingster, 1984). By involving unions during the plan's

development, management can gain insight into the areas which are of

particular concern to the unions and may be better able to design a plan that

suits management objectives and at the same time is acceptable to the union.

10
*.:-.

F'.

Table 3

Lack of Enthusiasm for Performance Appraisal

Performance appraisals are "often viewed with the same enthusiasm as income

tax forms" (McCall & DeVries, 1977, p. 1).

"It seems as if formal performance appraisal is not so much an accepted part

of the management scene as it is a tolerated one, something of a necessary

evil...However necessary some formal appraisal system appears to be, current

systems are still widely regarded as a nuisance at best and a dangerous evil

at worst" (Conference Board, 1979, p. I).

"Performance appraisal systems are a lot like seat belts. Most people

believe they are necessary, but they don't like to use them" (Latham &

Wexley, 1981, p. 2).

-The chief value thus far of performance evaluation to personnel management

seems to be that-it lends itself readily to theoretical rambling by writers,

statisticians and psychologists; its lot in the shop, office and warehouse

has been querulous--a burden to supervisors, a frustration to personnel

administrators, and a mystery to employees" (Dailey, 1961, p. 42).

"For the personnel practitioner, the use of the present PA system may

represent the path toward 'utter despair and hopelessness', while its

abandonment represents the path toward 'extinction'" (Bernardin & Beatty,

1984, paraphrasing Woody Allen, p. 2).

"Many appraisal systems are being severely criticized from all sides.

Managers find them troublesome, particularly when they have to criticize an

employee personally and put the criticism in writing. Employees charge that

the appraisals are often too subjective, and the federal courts frequently

agree with them. Most disappointing of all, many executives themselves

realize that existing performance appraisal systems do not bring about a

positive change in their employees' behavior" (Latham & Wexley, 1981,

p. 1-2).

.11
Table 4

Performance Appraisal Systems Seen as Failures

"The inaccuracy of the performance measurement system has led some authors to

conclude.. .performance appraisal[s] are a waste of effort. Other authors

have blamed the failure of appraisal systems on the way appraisals are

implemented" (Edwards, 1983, p. 13).

"There is abundant evidence that performance appraisal programs often fall to

produce the expected benefits of opening up lines of communications and

enhancing work effectiveness, in addition to providing administratively

useful appraisals" (Finn & Fontaine, 1984, p. 335).

"host organizations are dissatisfied with their performance appraisal (PA)

process, particularly for administrative/managerial positions. They have concerns about its objectivity, its relevance and its validity. In many cases the complaint is that the appraisal system simply does not work"

(Schneier & Beatty, 1979, p. 65).

[Performance appraisal) "is in a troubled state--it makes great theoretical

sense, but falters in actual practice" (DeVries et al., 1980, p. 11).

"Performance appraisal has created more consternation than perhaps any other

single issue in American management. Organizational approaches to

performance appraisal typically range from dismal to mediocre, with a very

few verging on the excellent" (Albrecht, 1978, p. 153).

"In general, the empirical work on appraisal could be entitled 'Another Thing

That Can Go Wrong with Performance Appraisal, Part 1 ...Infinity" (Bernardin

& Beatty, 1984, p. 2).

"No management tool has been given more lip service and used less effectively

than employee performance appraisals" (Beacham, 1979, p. 53).

"Performance appraisal systems do not and cannot possibly work.. .The emerging

problems of this new reform (CSRA), are not traceable to supervisory

carelessness, nor to the inevitable 'shakedown period' which follows any

major change, but are inherently a part of any appraisal system. If, as it

is argued, a given task is impossible to perform, no level of effort will

overcome that inherent impossibility" (Thayer, 1981, p. 21).

"Performance appraisal has remained an unsolved and perhaps unsolvable

problem in human-resource management...To be sure, many gimmicky innovations

have been devised for reviewing managerial and employee performance, yet they

have proven to be short-lived. The search for the Holy Grail in performance

appraisal goes on" (Patten, 1982, p. 1).

"In spite of the efforts of researchers and practitioners, performance

appraisal systems remain more of an albatross than an effective

organizational tool" (McCall & DeVries, 1977, p. ii).

12
I..

Top Management Backing is Essential

The importance of obtaining top management backing was asserted in the

literature (e.g., DeVries, et al., 1980; Henderson, 1984; Steers & Lee, 1983;

Steinberg & Burke, in press) and also emphasized by the subject-matter

experts. DeVries et al. (1980, p. 107) concluded tnat "lack of top

management support.. represents one of the most frequently cited reasons

for failure of PA systems." The subject-matter experts confirmed that

obtaining top management backing should be the first item on the agenda prior

to designing or revising a PA system and they cautioned that patience is

required in that effort. Though it might take up to 3 to 5 months of careful

marketing in order to obtain this top-level backing, the end result would be

worth it.

The reason for this strong emphasis on the role of top management is the

belief that this group can be instrumental in making PA work and in reversing

user apathy and resistance. Top management should set PA policy and reward

those who implement it. Top management should assign responsibility for PA

to a credible and visible key executive, allocate the resources necessary

to implement the system, identify organizational goals for linkage to

appraisals, and sponsor a campaign to gain acceptance for the appraisal

system from employees at all levels in the organization. These same

initiatives begun without top management support lack the necessary clout

and accordingly are not taken as seriously.

The Performance Appraisal Process Should be Used as a Management Tool

Both the literature and the subject-matter experts advocated that, in

addition to its use in making personnel decisions such as promotions and

awards, PA should be used as a tool for management to plan and review

performance, motivate employees and communicate job requirements and

evaluation criteria. This function of PA is clearly mandated by the CSRA

requirement to develop yearly individual performance plans which communicate

elements of the job and performance standards.

However, these sources also revealed that the PA process used as a means

to improve individual performance is infrequently practiced. More often than

not, performance appraisal is seen as a personnel department requirement and

an obstacle that takes the manager away from the important duties oi the job

(Godwin & Needham, 1981). Managers argue that PA:

o requires too much time (DeVries et al., 1980).

o does not help get the job done (Godwin & Needham, 1981).

o is not a good tool because it constitutes delayed feedback and, as

such, is not in accord with accepted learning principles (Dailey &

Madsen, 1980).

13

- •'

- - .4
o serves too many purposes. This results in feedback which is

designed to meet the requirements of personnel decisions, not

improvement of performance (McMillan & Doyel, 1980; Yager, 1981).

o is uncomfortable for both the supervisor and subordinate (Cohen &

Jaffee, 1982; Levinson, 1976).

One of the reasons for these objections is the view that PA is a

once-a-year act of filling out forms and confronting subordinates. Advocates

of the use of PA as a management tool, however, suggest that this should not

be so. Rather, they envision PA as part of an ongoing management process

(Baird, et al., 1982; Conference Board, 1979; Meyer, Kay, & French, 1965;

Yager, 1981). This process is diagrammed in Table 5.

Identification of objectives at the beginning of the appraisal period

(i.e., goal setting, see Step 1 in Table 5) is an integral part of developing

the performance plan. Setting specific goals has been shown to improve

performance. (See Latham and Yukl, 1975, for a review of goal setting in 27

organizations, and Locke, Shaw, Saari, and Latham, 1981, for a review of the

goal-setting literature from 1969 to 1980.) The goal-setting interview,

however, requires preparation on the part of both the supervisor and the subordinate. A useful guide for both is provided by Labovitz and Baird

(1982).

The supervisory role includes coaching, monitoring and documenting

performance, and providing feedback (see Step 2, Table 5). It is especially

important to provide continuous feedback throughout the appraisal period, and

no t only a t the end , because i t:

o provides information on goal accomplishment and demonstrates concern

about performance (Ilgen, Fisher, & Taylor, 1979).

o is less effective when delayed (Dailey & Madsen, 1980; Meyer et al.,

1965).

o improves the likelihood of employee acceptance because supervisory

suggestions for improvement do not overwhelm the subordinate by

being given all at once (Meyer et al., 1965).

o makes less likely an unexpected, emotional confrontation at PA time

(Henderson, 1984; Yager, 1981).

Based on a review of 16 studies, Landy, Farr, and Jacobs (1982) concluded

that feedback improves performance. Earlier, Landy, Barnes, and Murphy

(1978) found that identification of goals and frequency of evaluation are

both related to the perceived fairness and accuracy of performance

evaluation. A note of caution, however, is in order. Feedback does not

have uniformly successful results (Ilgen et al., 1979; Meyer, 1977). For

some, feedback may fail to improve performance and even be detrimental to

it. Very frequent feedback from supervisors, for example, may result in

14

.W.
M.~

Table 5

Steps of the Performance Appraisal Process

Step I

Supervisor and subordinate develop written performance plan at the

beginning of the appraisal period. The plan reflects organizational

goals and job requirements. It contains critical elements and

objective standards which will be the basis for the appraisal at

the end of the year.

Step 2

Supervisor monitors and documents performance of subordinate,

coaches subordinate, and provides feedback on performance.

Subordinate monitors and documents own accomplishments and

problems encountered.

Step 3

Supervisor conducts an interim review--supervisor and subordinate

compare performance on job elements to performance standards,

plan to improve performance if necessary, and revise performance plan

if necessary to take into account changing requirements or conditions.

Step 4

Repeat step 2.

Step 5

Repeat step 3, if needed.

Step 6

Supervisor makes appraisal and conducts the PA interview.

W,4

15

,.,., <.: ,, .., :. .. .: . o . .- . :. . . .
the perception of loss of control and this, in turn, may lead to excessive

reliance on others. The perception of the feedback and the response to it

are a function of the source of the feedback, the message, and the recipient.

(For a thought-provoking review on this literature, see Ilgen et al., 1979.)

Issues related to successful feedback should be dealt with extensively in

managerial training programs.

Self-monitoring and documenting of performance by the subordinate should

be encouraged by the supervisor. The subordinate thereby becomes part of the

process instead of a bystander (Ilgen et al., 1979). Also, self-monitoring

of results of past performance has been shown to improve future performance

(Landy et al., 1982).

The interim review (Step 3, Table 5) serves several functions: it tells

employees how they are doing; it provides the opportunity to plan for

improvement; and it allows reassessment of desired goals (Conference Board,

1979). Organizational goals and expectations do change and it is important

to communicate and clarify these mission changes (Bates, 1984), and update

the performance plan. However, experience shows that when the procedure to

modify the performance plan mid-cycle is too much trouble (i.e., requires

considerable paperwork or second-level approval), the plan is simply ignored.

Some sort of addendum attached to the performance plan may be one easy

solution.

The Performance Appraisal Process Should be Linked to Organizational Goals

The linkage of PA to organizational goals is highly recommended.

It is mandated by recent law (see above) and was advocated by both the

subject-matter experts and the literature (DeVries et al., 1980; Henderson,

1982; Odiorne, 1979). Though the concept appears to be fairly

straightforward, putting it into practice is not. It requires that

(1) organizational goals be identified by top management and communicated

throughout the organization, and that (2) PA be linked to the accomplishment

of these goals.

For PA purposes, mission statements which identify and describe the

reason for the existence of the organization are often much too general.

They need to be stated in terms of specific organizational objectives and

updated periodically so that work activities throughout the organization may

be directed toward the same objectives and coordinated. The process of

identifying objectives should start at the top of the organization and filter

down through the organizational hierarchy (Henderson, 1982; Lee & Zwerman,

1976; Odiorne, 1979). At each level, they need to be refined further into

intermediate objectives and into the tasks required to accomplish them. This

process has aptly been called "cascade of goals" (Raia, 1974). It is

designed to assure that work-unit goals interlock and support organizational

objectives and to allow the translation of organizational planning into

specific individual objectives and performance plans.

The identification of organizational goals and the linkage of PA to them

is an integral part of management by objectives (MBO), as originally proposed

by Peter DrucKer (1954). According to Drucker, a manager's performance

16
F.,

should be evaluated by its contribution to accomplishment of organizational

goals. He recommended that managers require subordinates to write a

manager's letter" twice a year. In it, subordinates should: (1) define the

objectives of their own jobs and that of their supervisor, (2) write their

performance standards, (3) identify major obstacles to the accomplishment of

objectives, and (4) indicate proposed activities to accomplish the objectives

(Abbott & Schuster, 1984).

The basic IBO approach has been incorporated into CSRA and the

performance contract is embodied in the CSRA-required performance plan.

It must contain the critical elements, and non-critical elements if any,

and performance standards for the job. Until the passage of the Civil

Service Retirement Spouse Equity Act of 1984, there was no requirement to

show how accomplishments were related to organizational goals. The law does

not, require, however, that there be any systematic coordination of

performance plans (such as the "cascade-of-goals" approach) to assure that

goals are addressed in an integrated fashion.

An example of the MBO process in action in the Federal government can be

found in the Department of Commerce. A letter from the Secretary of Commerce

to all employees of the agency is provided in Appendix A. This letter

communicates agency goals for FY 1985-1988 and full top management support

for the MBO planning and management system used to accomplish these goals.

The goals (provided in Appendix A, as well) are general enough to cut across

organizational lines and yet specific enough for their accomplishment to be

verified. A computerized system exists to track the "cascade of goals" and

identify individual accountability for their attainment.

The Performance Appraisal System Should be Adapted to Organizational Needs

The interviewees frequently expressed the need to adapt their PA systems

to special features and/or missions of their agencies. Adaptations in

response to particular missions have involved the type of job analysis and by

whom conducted, degree of uniformity of elements and standards, timing of

appraisals, and type and amount of training. Specific conditions mentioned

which might call for adaptations were specialized occupations, many (or few)

individuals in a given job series, amount of geographical dispersion of the

organization, many small units working in isolation from the rest of the

organization, organizational size (large or small), and cyclical calendar

events which impact on scheduling of performance appraisals.

Although legal constraints prescribe the main features of PA systems

under CSRA (see above), each organization has much flexibility of

implementation in areas such as training (how people are trained and by whom,

content and frequency of training), job analysis (how it is done and by whom,

development of generic elements and/or standards), and linkage to

organizational goals (how the goals are determined, communicated, and linked

to PA). Since it is recommended that PA be considered a part of the

management process, it is logical that managers, who will be the users and

promoters of the system, should be involved in its development and adaptation

to organizational needs.

17
System Perpetuation Should Be Planned

Another theme addressed by the literature and the subject-matter experts

was the need to place more emphasis on mechanisms which maintain the momentum

of newly initiated PA systems. Often after the initial introduction to the

system, managers hear no more about it from their supervisors, subordinates,

personnel department, and higher management (DeVries et al., 1980). With no

ongoing reminders, support, and encouragement, managers are not likely to use

the PA process on a continuing basis.

It is not surprising, therefore, that PA systems traditionally have had

very brief shelf lives. The initial enthusiasm for new systems soon turns

into disillusionment and the system is discarded shortly in favor of a new

one that supposedly will improve on the previous one (Conference Board,

1979). Some examples follow:

o The first formal PA system in the U.S. Federal government was

introduced in 1842 and died of half-hearted practice and apathy by

the end of the 1850s (Lopez, 1968). The next Federal PA system

wasn't begun until 1887.

o The U.S. Navy used 48 different fitness reports in the 91 years from

1865 to 1956 (Lopez, 1968).

o In a survey of almost 300 industrial firms, more than half reported

that their current PA systems had been developed within the last 3

years (Conference Board, 1979).

o In a survey of 18 major national and/or international corporations

in the private sector, Teel (1980) found that 6 had systems which

were in operation for less than a year and 9 were currently being

revised. Only 5 had systems that were in use over 5 years.

Additional reasons cited for the abandonment of PA systems include:

(a) changing legal requirements (b) user resistance, and/or (c) inflation of

ratings. User resistance may be related to the process of PA in general, or

to some specific feature of a system such as the rating format. Inflation cf

ratings, in which most people are rated at or near the top of the scale, most

probably reflects something other than the existence of superior groups of

employees (Bernardin, Orban, & Carlyle, 1981; Ilgen & Feldman, 1983). It may

reflect (a) the perception that the other raters in the organization base

ratings on these factors and that they must also if their subordinates are to

remain competitive for promotions and awards; (b) the feelings of the rater

about performance ratings in general; (c) the rater's ability to rate

accurately; and (d) the concern of the rater about the subordinate's response

and future cooperation.

It has been suggested that, in order to maintain interest in PA systems

beyond the initial period of introduction, organizations should discard the

mechanistic notion that such systems can be installed like pieces of new

hardware which continue to operate by themselves (Albrecht, 1978). At the

very least, a continued effort should be made: (a) to address the concerns

18

-. 4
* - - - .- - -- - . - - - - . - . -i. i- a r c . . V

of supervisors and subordinates about how the system will be and is being

used; (b) to conduct training to develop skills in the use of the system; and

(c) to commuucate top-level support of the system (Barrett, 1966). One way

to assure that this continued effort Is made might be to appoint key line

managers (and/or core groups, the number and level of which would depend on

the structure of the organization) to be responsible for the PA system and

its operation as part of the management process.

Training is Essential

Training in PA is recommended in the literature for a variety of

purposes. It can be used to teach managers: to observe behavior accurately;

to reduce rating errors (halo, leniency, contrast effects, similar-to-me

errors, central tendency, recency, latency, and stereotypy); to conduct

better appraisal discussions; to improve supervisory skills in coaching; to

increase knowledge of the performance appraisal system; and to improve

attitudes toward, and motivation to use, PA systems (Davis & Mount, 1984;

Decker, 1982; Dunnette & Borman, 1979; Glasgow, Simkins, & Guerrieri, 1981;

McIntyre, Smith, & Hassett, 1984; Pursell, Dossett, & Latham, 1980; Spool,

1978; Zedeck & Cascio, 1982). Formal training techniques reported in the

research literature include various combinations of: reading assignments,

lectures (brief and long), discussions, diary keeping, videotapes, rating

practice, feedback, self-paced instruction, computer-assisted instruction,

role playing, and behavior modeling. Topics that have been recommended for

inclusion in PA training programs are listed in Table 6. As can be seen

there, some topics primarily require learning facts whereas others require learning skills as well. Unfortunately, the literature on teaching methods

does not distinguish the methods that work best for each purpose.

Much research has been conducted on the effectiveness of training as

a means to reduce rater error. For example, Dunnette and Borman (1979)

concluded that errors can be reduced via training, and Spool (1978), in a review of 25 years of research literature on training, concluded that in

almost all of the studies he reviewed, training improved accuracy in

observing behavior. Yet there is other evidence that training does not

always reduce rater error. Davis and Mount (1984) found that training

managers did not affect halo or leniency tendencies, and Warmke and Billings

(1979) found that although training reduced errors on experimental ratings,

it had no effect on subsequent administrative ratings. What the Davis and

Mount, and Warmke and Billings studies had in common were ratings which were for actual use in the organization. The influence of purpose of rating on determining success of training to reduce rater error is illustrated in the

research of Zedeck and Cascio (1982). These researchers manipulated the

purpose of rating (either to recommend development, to award a merit raise, or to retain a probationary employee) and found that there was less p discrimination among ratees when the rating was for the purpose of deciding a

merit increase.

The findings that differential effects of training depend on purpose-of

the training also calls into question the usefulness of the many studies of

training using college students making ratings for research purposes. The

findings of these studies may be far less predictive of the results of

19

| BI
training in working organizations than studies in which the subjects are

actual supervisors making ratings for administrative purposes (Warmke &

Billings, 1979). In similar fashion, research based upon what Ilgen and

Favero (1985) call the "paper people paradigm" (in which raters are presented

with written information about a stimulus person) may not necessarily

- generalize to the way people respond to actual people. This paradigm does

not take into account PA as a complex, cyclical process which involves the

* selection, integration, and recall of information over time (Feldman, 1981).

It also does not take into account the social, economic, and professional

consequences of ratings and resultant personnel actions that must be lived

with and accommodated to by both the rater and ratee (McCall & DeVries,

1977).

Another factor to consider with respect to training to reduce rater

error is the criteria for determining the amount of rater error. Many

studies have found that training raters reduces rater error (Landy & Farr,

1983). However, in most of these studies, the criteria for determining rater

error involved inspection of the response distribution as opposed to

comparison of the ratings to true criterion scores. The few studies using

the latter approach indicate that training does not improve rater accuracy,

and even may decrease it (Bernardin, 1979; Bernardin & Pence, 1980).

Other research with respect to training has shown that training can

definitely increase knowledge of the PA system and can improve skills in

formulating developmental plans for subordinates (Davis & Mount, 1984;

Glasgow, Simkins, & Guerrieri, 1981). Interviewees from several Federal

agencies indicated that training for appraisal purposes and for performance

management, in general, was needed to make appraisal systems work, but that

* lack of resources for training was a problem. Several agencies, however,

* were able to commission consultants to design training materials (manuals and

videotapes) for them.

One approach to developing training materials for PA is the modular

approach (Bianco, 1984). The modular concept offers the advantages of

- allowing the combination, deletion, or rearrangement of portions of the

* training program to meet organizational needs. With the modular approach,

* programs can consist of core modules that apply agency-wide. These can be

supplemented with additional modules specific to unit/departmental

requirements, the particular audience (e.g., supervisor or non-supervisor),

and the purpose of training (introductory/orientation, in-depth, or

refresher).

One training program of special interest, because it illustrates an

application of several themes discussed above, is that of the U.S. Army

Materiel Command (AMC). AMC arranged with OPM to design and conduct a

. training workshop, specifically tailored to Army requirements, on teaching

• supervisors to develop performance standards and individual performance plans.

Top management clearly communicated its strong backing for this training.

The commanding General requested AMC commanders to recommend two candidates

each to receive the training and then conduct supervisory training at the

installation level (see Appendix B). One candidate was to be from the

20

;.. .". ...:i i-' .. iii *. , , - -*-- "-*. .-. # i- .. . -- -',: - ' .-
Table 6

Recommended Performance Appraisal Training Topics

To Provide Information To Improve Skills

Definition of terms Developing performance plans

(PA cycle, elements, (identifying critical

critical elements, elements, linking employee

performance standards, objectives of organizational

performance plans, etc.) goals, developing objective

standards, etc.)

Purposes of performance Documenting performance

appraisal (diary, critical incidents,

etc.)

Legal requirements Conducting informal day-to-day

coaching and feedback

Organizational requirements Reducing rater error

(regulations, forms, due

dates, etc.)

Preparing for and conducting

interim reviews

Preparing for and conducting

appraisal interviews

21

I
+ - ., .. . . ..

Civilian Personnel Office (CPO) staff and one from line management. By

sending managers to the workshop in addition to personnelists, top management

indicated that managers have an important role in insuring the success of the

PA system. Also, system perpetuation was fostered by training supervisors to

be on-site experts and giving them responsibility for training others to make

the system work. To facilitate the training role, workshop participants were

provided with (a) a detailed Instructor's Manual for their use in delivering

instruction back home, including copies of overhead projector slides, (b) a

training plan for conducting the Instructor Briefing sessions, (c) a

camera-ready copy of a Participant's Coursebook to be used in conducting

sessions of the course, and (d) an evaluation instrument for use in obtaining

participant feedback on sessions.

Ease of Use is Important

Many Federal agencies are searching for ways to meet user demands for PA

systems that are easier to apply than those currently in use. Of particular

concern are two issues: the amount of paperwork and difficulty involved in

developing performance plans.

Proposed solutions for reducing the paperwork burden focus on the appraisal

forms. The solutions include simplifying the forms to make them easier to

complete, combining them so that the same form can be used for more than one

pay category, making them word-processor compatible to reduce typing required,

and permitting attachments to the forms to reflect changes in requirements

during the rating period rather than rewriting the entire document (see

Steinberg & Burke, in press).

The solutions considered for reducing the difficulty in developing

performance plans can be categorized into one of two approaches. The first

provides more and better training for the supervisor to develop individual

performance plans (e.g., see the discussion of the AMC training course,

above). The second approach involves developing performance plans (or

portions of them) that can be used for a specified group of employees, and

which, consequently, need not be developed separately for each one. This is

accomplished either by retaining the performance plan format but developing

generic elements and/or standards which apply to a whole group of employees,

or by using a different rating format, such as a rating scale, to communicate

elements and standards. The two alternatives of this second approach are

frequently considered because they relieve the individual supervisor of much

of the burden of developing performance plans. In addition, the rating-scale

alternative has received considerable attention in the literature. For these

reasons, both alternativeb will be presented in more detail below.

Gent.ric Elements and Standards. Since CSRA requires that the PA elements

and sta Jards relate to the given job, development of generic elements and

standards (i.e., ones which apply to a whole group of employees) should be

based on a job analysis of the cluster of jobs covered. For large groups of

employees, this may require outside consultants and can be very costly and

time-consuming.

22
One set of generic elements and/or standards can cover a few related job

series within an agency (e.g., secretarial/clerical positions), or it may

apply to a much larger portion of the organization (e.g., all non-PMRS,

non-SES employees). One disadvantage of generic standards is that as the

number and types of positions that are covered by the same set of standards

increase, the standards become less job-specific. Although the standards

may still be appropriate, they would be less likely to mention specific

job-related projects and due dates, for example. To address this issue, PA

systems may provide for generic standards which can be modified by the

addition of individual work plans, or permit unique elements and standards to

be added to model performance plans. In this way, more specific expectations

can be communicated to subordinates and the performance plans can be modified

to reflect changes in requirements. It also means, however, that more of the

burden of developing performance plans is returned to the supervisor.

Rating Scale Formats. Rating scales have a number of features that may

make them seem, at least on the surface, even more desirable than generic

elements and standards. Since a scale can be used for a large number of

employees, it too removes the burden of developing the appraisal instrument

from the individual supervisor. (Scales also require a job analysis of

positions they cover.) In addition, scales appear simpler to use and easier

to understand than lengthy performance plans. Also, they have been popular

formats for PA for some time, and continue to be used widely (DeVries et al.,

1980). For example, during the fall of 1981, Tyer (1982) conducted a survey

of the 50 state governments and found that rating scales for PA purposes are

still popular--they were used by at least half of the state governments.

There are, however, some disadvantages to using rating scales in a

Federal PA system. First, various features of PA scales, in their current

formats, are inconsistent with CSRA requirements. Second, scales are costly

and time-consuming to develop. These two issues are addressed in greater

detail below because:

(a) Although the PA literature is replete with descriptions of various

types of rating scales and research conducted on them (e.g.,

Bernardin & Beatty, 1984; Landy & Farr, 1980; and Schwab, Heneman,

& DeCotiis, 1975), there exists no systematic review of their

ability to meet CSRA requirements, and

(b) Although, many subject-matter experts suggested that rating scales

may be easier to use than individual performance plans, they were

not aware of the costly and time-consuming process to develop them.

In order for rating-scale formats to meet CSRA specifications, they must

contain objective, job-related standards (anchors) defined at the beginning

of the rating period, against which job performance can be compared. A

discussion of the more popular formats reported in the literature, and how

they stand with respect to these criteria, follows.

23 [ •

- -. , - .+.+:.+ *': .•.' ..". .- . ' -" -.- '-+." ." - .-.. . :.'"- ., - ". ., -".r '+' . " ".: : """, +" " : +
"

o Graphic rating scales are scales, each of which represents the range V

of a given trait or factor (such as motivation, cooperation, or job

knowledge). The rater indicates how much of the factor the ratee

has (DeVries et al., 1980). The scale may be a continuous line or

it may be divided into several discrete categories anchored by

numbers or a series of adjectives such as "low" through -high" or

-"below average" to "above average" (McCormick & Ilgen, 1980;

Muchinsky, 1983; Siegel & Lane, 1982). These anchors do not provide

the CSRA-required specific job-related behavioral definitions. As

such, they do not provide an objective basis for the rater to select

a given rating, nor do they convey to ratees the basis upon which

they will receive a given rating.

o Employee comparison formats include rank-order scales, pairedcomparison scales and forced-distribution scales (Muchinsky, 1983).

These all provide for the rating of individuals by comparing them

to each other, instead of to a pre-defined performance standard as

required by CSRA. Rank-order scales require the ranking of

employees from high to low; paired-comparison scales require

that each employee be compared to every other one; and forceddistribution scales require that the ratings conform to a preset

distribution.

o Forced-choice scales, introduced by Robert Wherry in the early 1940s

(and used at one time for Army officer ratings, Sisson, 1948), are

designed to prevent rater attempts to distort ratings in favor of or

against a given individual. They do this by masking the weightings

assigned to each statement so that the raters cannot be certain which

is more favorable (Bernardin & Beatty, 1984). This clearly would not

be compatible with the CSRA requirement to communicate standards at

the beginning of the rating period.

o Mixed-standard scales also attempt to prevent rater manipulation by

disguising the scoring procedure (Rosinger, Meyers, Levy, Loar,

Mohrman, & Stock, 1982). Statements reflecting three degrees of each

trait or behavior to be rated are all intermixed so that both the

dimension that each statement represents and the order-of-merit of

the triad are not obvious (Bernardin & Beatty, 1984; Blanz &

Ghiselli, 1972). In other words, the elements and standards would be

masked in contravention of CSRA specifications.

o Behavioral observation scales (BOS) consist of lists of behavioral

statements, derived from a critical incident job analysis, which are

all rated on one five-point Likert scale for frequency with which

the ratee engages in the stated behavior (Latham, Fay, & Saari,

1979). The behavioral statements may be grouped into dimensions

(i.e., elements, in CSRA terms). The ratings on each of the

behavioral statements are summed to determine the overall score

for each dimension, and these, in turn are summed to determine the

overall evaluation score. Management determines the range of scores

that correspondb to each level of the performance standards.

24

"'- .- _" ,,.:r ,'.., :..'":" _,".-"- " : -: .' . . . " .. ". . . .. ' . : . ". . .. . . .. . . .. .' -
The five-point frequency scale on which the behavioral statements

are rated may have two anchors, such as "almost never" and "almost

always" at either end, or it may have anchors at each point which

indicate the percentage of time that the listed behavior occurs when

required..

Two features of BOS scales which may be inconsistent with CSRA

requirements are presented below:

- The same frequency scale is used for all task statements.

However, identical numerical ratings for different task statements

do not necessarily represent the same level of satisfactory

achievement (Bernardin & Kane, 1980). Therefore their translation

to performance standard levels is inappropriate. To illustrate

this point, consider the following example. Here are two tasks

from a BOS for an airplane mechanic and the scale on which they

are rated:

I = 0 - 64% of the time

2 = 65% - 74Z of the time

3 = 75% - 84% of the time

4 = 85% - 94% of the time

5 = 95% - 100% of the time

--Keeps an orderly tool box.

-- Reports life-threatening problems promptly.

It is conceivable that keeping an orderly tool box 80% of the

time might be considered satisfactory, but reporting airplane

mechanical problems promptly 80% of the time might not be.

Yet, using the above scale, both behaviors would receive the

identical rating of "3." Further, when scores on items such as

these are summed, the score does not indicate whether the total

of "6," for example, resulted from a score of -2" on the first

item and "4" on the second, or the other way around.

- items which are observed very frequently or very infrequently

are not included in the scale because they do not differentiate

between good and poor performers (Latham et al., 1979).

Therefore, many behaviors which are required for the job, and

even some that are critical to it, do not enter into the rating.

0 Behaviorally anchored rating scales (BARS) originally referred to as

behavioral expectation scales (BES), have behavioral anchors based

on critical incidents aud clearly indicate the weight of each.

However, the anchors do not represent standards which must be met.

Rather, they are behaviors which the raters are to infer could be

expected to occur, given past ratee behaviors. Raters are to make

these inferential leaps from actual behaviors to expected behaviors

based upon their own implicit beliefs about how people behave

25

14

€1
R-K7: -4 - 777 T 7 -- - - -.-

(Nathan & Alexander, 1985; Smith & Kendall, 1963). Thus, the CSRA

requirement of providing, at the beginning of the rating period,

standards against which job performance can be compared, would not

be met.

In their original article presenting BARS, Smith and Kendall (1963)

suggested the use of general statements representing high, low, and

acceptable levels of performance in addition to the scale anchors,

in order to help raters determine the appropriate rating. However,

they expressed concern that these statements, in addition to the

behavioral anchors, would result in ambiguous ratings. Although

some research supports the desirability of using these general

statements (Bernardin, LaShells, Smith, & Alvares, 1976), they are

rarely included in BARS. In fact, standard texts in the field of

personnel psychology do not even mention them. For example, although

Cascio (1982), Landy (1985), McCormick and Ilgen (1980), Miner and

Miner, (1977), Muchinsky (1983), and Siegel and Lane (1982) spend

from one and one half to three and one half pages each on BARS and

the detailed procedures to develop them, none mentions the general

sta temen ts.

The fact that the general statements of levels of performance are

similar to performance standards required by CSRA (see Bernardin &

Beatty, 1984, p. 84 and p. 87 for two examples), raises the

possibility that BARS meet the requirements of CSRA. However, the

issue of whether the "could-be-expected-to" anchors could be used

still remains.

In addition to the deficiencies in meeting CSRA requirements discussed

above, a second factor should be taken into account by those who hope to find

scales an easy solution. The development, use, and/or scoring of scales can

be much more difficult than would seem at first glance. Although the

following discussion illustrates these points using BARS examples, similar

problems are encountered with other scales as well.

o More than a mere check-mark on the scale may be required of the

rater. The recording of incidents of observed ratee behavior during

the rating period (three or more incidents for each of about 5 to 10

scales, for each ratee) is recommended for BARS (Bernardin et al.,

1976; Smith & Kendall, 1963).

o Many people must be involved in the development of scales. For

BARS, Smith and Kendall (1963), for example, used 542 head nurses;

Bernardin (1977) used 118 students; and Landy, Farr, Saal, & Freytag

(1976) used 168 supervisors to develop one particular BARS

instrument and 240 peers to develop another.

o Many critical incidents must be developed before the anchor points

on the BARS scale can be set. Goodale and Burke (1975), for

example, collected 360 incidents which they then narrowed down to 6

or 7 for each of 10 dimensions.

26

A
o The scale development procedure is an arduous, iterative process

which is costly and time-consuming (Bernardin & Beatty, 1984; Landy

& Farr, 1980).

o Because of the long and expensive procedure to develop BARS, the .

scales are not easily adapted to changing job requirements.

o Because there is no procedure in the development of a BARS to ensure

that the anchors for a dimension are mutually exclusive, more than

one anchor may apply to a given individual. For example, on a BARS

scale presented in Cascio (1982, p. 329), four of the nine ancoor

statements could conceivably apply to the same individual:

Scale Value Anchors

2.7 "Could be expected to go back on a promise

to an individual who he had told could

transfer back into previous department if

she/he didn't like the new one."

5.0 "Could be expected to remind sales personnel

to wait on customers instead of conversing

with each other."

7.0 "Could be expected never to fail to conduct

training meetings with his people weekly at

a scheduled hour and to convey to them

exactly what he expects."

8.5 "Could be expected to conduct a full day's

sales clinic with two new sales personnel

and thereby develop them into top sales

people in the department."

o Rating format development variations with BARS can impact on the

degree of psychometric error and, therefore, users who might be

tempted to vary the established procedures (perhaps to simplify

them), should be cautioned not to do so without evaluating the

impact. For example, Rumsey and Nietus (in press) found that the

absence of features such as rating clarification statements,

grouping of items by dimension, and overall evaluation by dimension,

can significantly reduce rating accuracy.

27

r . . .
CONCLUSIONS

Nine prescriptive themes for more effective Federal PA systems emerged from a review of the literature and interviews with subject-matter experts. These

themes are based on opinion as well as empirical evidence, but they represent

the collective wisdom of those knowledgeable in the field.

The themes discussed above were presented one by one, but in practice,

they require integration to make the PA system work. Addressing any one, to

the exclusion of others, lessens the chances for success. For example, the

recommendation to make the system easy to use was expressed so frequently by

interviewees that it was tempting to present it first in the discussion of

themes instead of last. Yet, the first two recommendations were to ensure that the PA system meets legal requirements and that it is seen as important

and worthwhile by the users. If the system is easy to use but not legal, it

is useless, and if users see no purpose or advantage to PA, any amount of

effort devoted to it will be perceived as too much.

Therefore, the first theme emphasized the primary reason for the current

structure of PA systems in the Federal sector--it is required by law.

However, the mere existence of a system which meets legal specifications is

insufficient to ensure usage beyond minimal compliance. Thus, the themes

which followed emphasized the steps that are needed to make the system work

as intended:

o As with any new program that requires the participation of many, the

program must be -sold.- With PA this step is especially crucial.

It is highly likely that many users at all levels of the

organization already have negative feelings about the process and

its worth.

o In selling the program, it is important to start at the top of the

organization because top management plays a vital role in ensuring

the program's success. Top management must both communicate its

commitment to developing and maintaining an effective PA system,

and obtain the commitment of the remainder of the organization.

Top management should assign the responsibility for promoting the

PA system to a respected, high-level line manager who can devote

considerable time to ensuring that the system be made to work. The

emphasis is on line-management sponsorship over personnel department

sponsorship in order to establish system credibility with

lower-level managers and supervisors.

o In order for supervisors to be sold on the PA process, they must come to view it as an integral part of their jobs. They must

understand that PA is part of a year-round management process which

includes identification of goals, planning to accomplish them,

coaching, monitoring and documenting performance, and providing

feedback.

28

'"-:"- ". - -" . .. " --" " "- ": ,-.: -. ", •" " ','': :: " ':':; F : ": "i:: : :: : : W4
pm

. o 

L

L o The next step is for top management to identify specific

organizational goals or objectives (to be accomplished in a specified

period of time, such as 1 to 4 years) with measurable outcomes. And,

it should emphasize the linkage of PA to the accomplishment of these

goals. Supervisors need to communicate to their subordinates the

connection between the work they are assigned to do and the

accomplishment of these organizational goals.

o Unless the PA system is designed to take into account idiosyncrasies

of the organization such as size, dispersion, and diversity,

supervisors and managers will have a hard time accepting that the

system is supposed to help them accomplish organizational goals.

Adaptation of the PA system to organizational needs is important.

Managers and supervisors should take an active role in this process.

o The PA system should be designed so that commitment to it will be

maintained beyond the initial introductory stage. This requires

on-site managers who demonstrate by example their commitment to the

system, and train other managers in the skills required to plan,

coach, and provide feedback on performance.

o Training should be conducted for a variety of purposes (e.g., to teach

knowledge of the system, to teach skills such as planning, coaching,

and providing feedback) and for different audiences (e.g.,

supervisors, subordinates, military supervisors of civilians). The

majority of the training should be conducted in-house by appropriate

role models. Managers and supervisors, as opposed to personnelists,

should conduct most of the training.

o Finally, the system should be made as easy to use as possible.

Although at first glance, the above recommendations may appear to

entail even more effort than is commonly exerted to make PA syste:is

work, they also may have the effect of easing some of the problems.

When supervisors know that the time they spend on the PA process is

valued by top management as part of the job, they will be less

resentful of the time it takes. If they find the process can be

useful in facilitating the accomplishment of organizational goals,

they will be further motivated to make it work. And, with

appropriate training, they will find the system easier to use.

Further, such features as a simplified means for updating

performance plans and cooperative development of model performance

plans should ease the burden as well.

In sum, the advice of Glickman given in 1964 still applies: If you are

searching for a PA system that is easily understood, does not require much

time or training to use, and which the appraiser generally feels comfortable

using, then "you have described a relatively superficial and ineffective

performance appraisal system--one which is not considered very important by

the parties involved--one which could be eliminated without any great

deterioration in the personnel management program of which it was a part. A

good performance appraisal, one that provides useful information to the

29
employee, to his supervisors, and to management, takes considerable effort on the part of all of these parties to develop and use effectively. .. Thus, In an appraisal system, as in most human enterprises, you get what you are willing

to pay for--there are no cheap but practical performance appraisal systems"*

(pp. 28-29).

30

• -. - -. . ..-..- ". .. - - :-- -.. -- ....-' -i--i - .- - . '..- . i ,- - . " -. . . ,..
References

Abbott, J. R., & Schuster, F. S. (1984). History and theory of performance

appraisal. In M. Rock (Ed.), Handbook of wage and salary administration.

(pp. 21/3-21/26). New York: McGraw-Hill.

Albrecbt, K. (1978). Successful management by objectives: An action manual.

Englewood Cliffs, NJ: Prentice-Hall."

Baird, L., Beatty, R. W., & Schneier, C. E. (1982). The performance

appraisal sourcebook. Amherst, MA: Human Resource Development Press.

Barrett, R. S. (1966). Performance rating. Chicago: Science Research

Associates.

Bates, T. R. (1984). Managing a cast of thousands--Merit Pay employees.

Management, 4(3), 22-25.

Beacham, S. T. (1979). Managing compensation and performance appraisal under

the Age Act. Management Review, 68(1), 51-54.

Bernardin, H. J. (1979). Rater training: A critique and

reconceptualization. Academy of Management Proceedings, 32, 216-220.

Bernardin, H. J., & Beatty, R. W. (1984). Performance appraisal: Assessing

human behavior at work. Boston: Kent Publishing Co.

Bernardin, H. J., & Kane, J. S. (1980). A second look at behavioral

observation scales. Personnel Psychology, 33, 809-814.

Bernardin, H. J., LaShells, M. B., Smith, P. C., & Aivares, K. M. (1976).

Behavioral expectation scales: Effects of developmental procedures and

formats. Journal of Applied Psychology, 61, 75-79.

Bernardin, H. J., Orban, J. A., & Carlyle, J. J. (1981). Performance ratings

as a function of trust in appraisal, purpose of appraisal, and rater individual differences. Academy of Management Proceedings, 311-315.

Bernardin, H. J., & Pence, E. C. (1980). Effects of rater training:

Creating new response sets and decreasing accuracy. Journal of

Applied Psychology, 65, 60-66.

Bianco, V. (1984). In praise of performance. Effects of rater training: Creating new response sets and decreasing accuracy. Personnel Journal,

63(6), 40-50.

Blanz, F., & Ghiselli, E. E. (1972). The mixed standard scale: A new

* rating system. Personnel Psychology, 25, 189-199.

31

. • . " .**" " . " " * * " "-
Cascio, W. F. (1982). Applied psychology in personnel management. Reston,

VA: Reston Publishing Company.

Civil Service Reform Act of 1978. (1978). United States Code Congressional

and Administrative News, 1, 1111-1227.

Civil Rights Act of 1964. (1964). United States Code Congressional and

Administrative News, 1, 287-319.

Civil Service Retirement Spouse Equity Act of 1984. (1984). United States

Code Congressional and Administrative News, 2, 3195-3220.

Cohen, S. L., & Jaffee, C. L. (1982). Managing human performance for

productivity. Training and Development Journal, 36(12), 94-100.

Conference Board (1979). Appraising managerial performance: Current

practices and future directions. New York: The Conference Board Inc.

Dailey, C. A., & Madsen, A. M. (1980). How to evaluate people in business.

New York: McGraw Hill.

Dailey, W. W. (1961). Needed: A new manifesto for performance evaluation.

Personnel Administration, 24(4), 41-46.

Davis, B. L., & Mount, N. K. (1984). Effectiveness of performance appraisal

training using computer assisted instruction and behavior modeling.

Personnel Psychology, 37, 439-452.

Decker, P. J. (1982). The enhancement of behavior modeling training of

supervisory skills by the inclusion of retention processes. Personnel

Psychology, 35, 323-332.

. Department of the Army. (1985, January 30). Officer evaluation reporting

," system. Officer Ranks Personnel UPDATE, Issue No. 3, AR 623-105, pp. 1-62.

DeVries, D. L., Morrison, A. M., Shullman, S. L. & Gerlach, N. L. (1980).

Performance appraisal on the line. Greensboro, NC: Center for Creative

Leadership.

Drucker, P. F. (1954). The practice of management. New York: Harper & Row.

Dunnette, M. D., & Borman, W. C. (1979). Personnel selection and

classification systems. Annual Review of Psychology, 30, 477-525.

Edwards, M. R. (1983). Productivity improvement through innovations in

performance appraisal. Public Personnel Management, 12, 13-24.

Eitelberg, M. J., Laurence, J. H., Waters, B. K., & Perelman, L. S. (1984).

Screening for service: Aptitude and education criteria for military

entry. Alexandria, VA: Human Resources Research Organization.

32

. .

. . . . . . . I. . . . . . .. . . . .
Feldman, J. M. (1981). Beyond attribution theory: Cognitive processes

in performance appraisal. Journal of Applied Psychology, 66, 127-148.

Finn, R. H., & Fontaine, P. A. (1984). Performance appraisal: Some dynamics

and dilemmas. Public Personnel Management, 13, 335-343.

Gaertner, G. H., Gaertner, K. N., & Akinnusi, D. M. (1984). Environment,

strategy, and the implementation of administrative change: The case of

civil service reform. Academy of Management Journal, 27, 525-543.

Glasgow, Z., Simkins, N. L., & Guerrieri, J. A. (1981). Job performance

appraisal system rating program (AFHRL-TR-80-56). Brooks Air Force Base,

TX: Air Force Human Resources Laboratory.

Glickman, A. S. (1964). Is performance appraisal practical? Personnel

Administration, 27(5), 28-32.

Godwin, P., & Needham, J. (1981). Reforming reform--Challenging the

assumptions for improving public employees' performance. Public Personnel

Management, 10, 233-243.

Goodale, J. G., & Burke, R. J. (1975). Behaviorally based rating scales need

not be job specific. Journal of Applied Psychology, 60, 389-391.

Gordon, M. E., & Johnson, W. A. (1982). Seniority: A review of its legal

and scientific standing. Personnel Psychology, 35, 255-280.

Hall, D. T. (1983). The effect of the individual on an organization's

structure, style, and process. In F. Landy, S. Zedeck, & J. Cleveland

(Eds.), Performance measurement and theory (pp. 11-30). Hillsdale, NJ:

Lawrence Erlbaum Associates.

Henderson, R. (1982). Developing performance measures which are consistent

with the mission of the organization. In L. Baird, R. W. Beatty, & C. E.

Schneier, The performance appraisal sourcebook (pp. 17-24). Amherst, MA:

Human Resource Development Press.

Henderson, R. I. (1984). Practical guide to performance appraisal. Reston,

VA: Reston Publishing Co.

Heneman, H. G., III, Schwab, D. P., Fossum, J. A., & Dyer, L. D. (1983).

Personnel/human resource management. Homewood, IL: Richard Irwin.

Ilgen, D. R., & Favero, J. L. (1985). Limits in generalization from

psychological research to performance appraisal processes. Academy of

Management Review, 10, 311-321.

Ilgen, D. R., & Feldman, J. M. (1983). Performance appraisal: A process

focus. In L. L. Cummings & B. M. Staw (Eds.), Research in organizational

behavior (pp. 141-197). Greenwich, CT: JAI Press.

33
Ilgen, D. R., Fisher, C. D., & Taylor, M. S. (1979). Consequences of

individual feedback on behavior in organizations. Journal of Applied

Psychology, 64, 349-371.

Ingster, B. (1984). Appraising hourly performance. In M. L. Rock (Ed.),

Handbook of wage and salary administration (pp. 22/1-22/9). New York:

McGraw-Hill Book Company.

Kirkpatrick, D. L. (1984). Two ways to evaluate your performance appraisal system. Training and Development Journal, 38(8), 38-40.

Kleiman, L. S., & Durham, R. L. (1981). Performance appraisal, promotion,

and the courts: A critical review. Personnel Psychology, 34, 103-121.

Labovitz, G., Baird, L. S. (1982). MBO as an approach to performance

appraisal. In L. Baird, R. W. Beatty, & C. E. Schneier, The performance

appraisal sourcebook (pp. 51-56). Amherst, MA: Human Resource

Development Press.

Landy, F. J. (1985). Psychology of work behavior. Homewood, IL: The

Dorsey Press.

Landy, F. J., Barnes, J. L., & Murphy, K. R. (1978). Correlates of perceived

fairness and accuracy of performance evaluation. Journal of Applied

Psychology, 63, 751-754.

Landy, F. J., & Farr, J. L. (1980). Performance rating. Psychological

Bulletin, 87, 72-107.

- Landy, F. J., & Farr, J. L. (1983) The measurement of work performance:

* Methods, theory, and applications. New York: Academic Press.

Landy, F. J., Farr, J. L., & Jacobs, R. R. (1982). The bottom line in

performance evaluation. Public Productivity Review, 6, 78-92.

Landy, F. J., Farr, J. L., Saal, F. E., & Freytag, W. R. (1976).

Behaviorally anchored scales for rating the performance of police officers.

Journal of Applied Psychology, 61, 750-758.

Latham, G. P., Fay, C. H., & Saari, L. M. (1979). The development of

behavioral observation scales for appraising the performance of foremen.

Personnel Psychology, 32, 299-311.

Latham, G. P., & Wexley, K. N. (1981). Increasing productivity through

* performance appraisal. Reading, MA: Addison-Wesley Publishing Co.

* Latham, G. P., & Yukl, G. A. (1975). A review of research on the application

of goal setting in organizations. Academy of Management Journal, 18,

824-845.

34
Lee, M. B., & Zwerman, W. L. (1976). Designing a motivating and team

building employee appraisal system. Personnel Journal, 55, 354-356.

Levinson, H. (1976). Appraisal of what performance? Harvard Business

Review, 54(4), 30-46, 160.

Locke, E. A., Shaw, K. N., Saari, L. M., & Latham, G. P. (1981). Goal

setting and task performance: 1969-1980. Psychological Bulletin, 90,

125-152.

Lopez, F. N., Jr. (1968). Evaluating employee performance. Chicago:

Public Personnel Association.

Lubben, G. C., Thompson, D. E., & Klasson, C. R. (1980). Performance

appraisal: The legal implications of Title VII. Personnel, 57(3),

11-21.

McCall, M. W., Jr., & DeVries, D. L. (1977). Appraisal in context: Clashing

with organizational realities. Greensboro, NC: Center for Creative

Leadership.

McCormick, E. J., & Ilgen, D. R. (1980). Industrial psychology. Englewood

Cliffs, NJ: Prentice-Hall.

McIntyre, R. M., Smith, D. E., & Hassett, C. E. Accuracy of performance

ratings as affected by rater training and perceived purpose of rating.

Journal of Applied Psychology, 69, 147-156.

McMillan, J. D., & Doyel, H. W. (1980). Performance appraisal: Match the

tool to the task. Personnel, 57(4), 12-20.

Meyer, H. H. (1977). The annual performance review discussion--Making it

constructive. Personnel Journal, 56, 508-511.

Meyer, H. H., Kay, E., & French, J. R. P., Jr. (1965). Split roles in

performance appraisal. Harvard Business Review, 43(1), 123-129.

Miner, J. B., & Miner, M. G. (1977). Personnel and industrial relations:

A managerial approach. New York: Macmillan.

Muchinsky, P. M. (1983). Psychology applied to work: An Introduction to

industrial and organizational psychology. Homewood, IL: The Dorsey Press.

Nathan, B. R., & Alexander, R. A. (1985). The role of inferential accuracy

in performance rating. Academy of Management Review, 10, 109-115.

Odiorne, G. S. (1979). MBO II: A system of managerial leadership for

the 80s. Belmont, CA: Fearon Pitman Publishers.

35

I7-
Office of Personnel MIa.nagement. (1979). Federal Register, 44(151),

45587-45591.

Office of Personnel Management. (1985). Federal Register, 50(57),

11788-11799.

Patten, T. H., Jr. (1982). A manager's guide to performance appraisal:

Pride, prejudice and the law of equal opportunity. New York: The

Free Press.

Pelissero, J. P. (1984) Personnel evalua ion and the military manager:

Contrasts in performance appraisal systems. Public Personnel Management

13, 121-132.

Performance rating. (1977). 5 U.S.C.A. § 4301.

Pursell, E.D., Dossett, D. L., & Latham, G. P. (1980). Obtaining validated

predicttors by minimizing rating error in the criterion. Personnel

Psychology, 33, 91-96.

Rala, A. P. (1974). Managing by objectives. Glenview, IL: Scott, Foresman

and Company.

Rosinger, G., Myers, L. B., Levy, G. W., Loar, M., Mohrman, S. A., &

Stock, J. R. (1982). Development of a behaviorally based pe-formance appraisal system. Personnel Psychology, 35, 75-88.

Rumsey, M. G., & Mietus, J. R. ( 1984 ). A comparison of two behaviorally

anchored rating scales (Technical Report No. 626). Alexandria, VA: U.S.

Army Research Institute. AD A153230

Schneier, C. E., & Beatty, R. W. (1979). Integrating behaviorally-based and

effectiveness-based methods. Personnel Administrator, 24(7), 65-76.

Schwab, D. P., Heneman, H. G., II, & DeCotiis, T. A. (1975). Behaviorally

anchored rating scales: A review of the literature. Personnel

Psychology, 28, 549-562.

Siegel, L., & Lane, I. M. (1982). Personnel and organizational psychology.

Georgetown, Ontario: Richard Irwin.

Sisson, E. D. (1948). Forced choice - The new Army rating. Personnel

Psychology, 1, 365-381.

Smith, P. C., & Kendall, L. M. (1963). Retranslation of expectations: An

approach to the construction of unambiguous anchors for rating scales.

Journal of Applied Psychology, 47, 149-155.

Spool, M. D. (1978). Training programs for observers of behavior: A reviev.

Personnel Psychology, 31, 853-888.

36
Steers, R. M., & Lee, T. W. (1983). Facilitating effective performance

appraisals: The role of employee commitment and organizational climate.

In F. Landy, S. Zedeck, & J. Cleveland (Eds.), Performance measurement and

theory. (pp. 75-88). Hillsdale, NJ: Lawrence Erlbaum Associates.

Steinberg, A. G., & Burke, W. P. (in press). A survey of performance

appraisal systems in Federal agencies. Alexandria, VA: U.S. Army

Research Institutefor the Behavioral and Social Sciences.

Teel, K. S. (1980). Performance appraisal: Current trends, persistent

progress. Personnel Journal, 59, 296-301.

Thayer, F. C. (1981). Civil service reform and performance appraisal: A

policy disaster. Public personnel Management, 10, 20-28.

Tyer, C. B. (1982). Employee performance appraisal in American state

governments. Public Personnel Management, 11, 199-212.

Uniform guidelines on employee selection procedures. (1978). Federal

Register, 43(166), 38290-38315.

United States Code Service (Lawyers Edition, Titles 1-18, January, 1985).

Rochester, NY: The Lawyers Co-operative Publishing Company.

Warmke, D. L., & Billings, R. S. (1979). Comparison of training methods for

improving the psychometric quality of experimental and administrative

performance ratings. Journal of Applied Psychology, 64, 124-131.

Yager, E. (1981). A critique of performance appraisal systems. Personnel

Journal, 60, 129-133.

Zedeck, S., & Cascio, W. F. (1982). performance appraisal decisions as a

function of rater training and purpose of the appraisal. Journal of

Applied Psychology, 67, 752-758.

37
APPENDIX A

EXAMPLE OF AGENCY STATEMENT OF GOALS AND SUPPORT FOR MBO

~~THE SECRETARY OF COMMERCE

Weshngton. D.C. 20230

VAR 2 9 1985

MF40ERANDUM FOR ALL EMPLOYEES

SUBJECT: Departmental Goals for FY 1985-1988

For the past year, Deputy Secretary Brown and the Department's managers have been

assessing how well we have done in meeting goals and objectives established for

the first four years of the Administration and considering the need for new and

*' revised goals. I am pleased to provide you with the results of that process: a

listing (attached) of new major Department of Commerce goals and subgoals for the

period 1985-1988.

These new goals accomplish several important things:

o They restate Departmental policies and priorities. Some of the goals we set

four years ago have been achieved; others have been overtaken by external

events. Now there are new goals that we can expect to achieve with the

* authorities that we have and the resources we can expect to get.

o They state our alms more specifically than the earlier goals.

o They cut across organizational lines more than before. This will improve communication and coordination among operating units with common interests

and responsibilities.

o They cover more programs than the earlier goals. Four years ago goals were

set only for those areas where major changes were envisioned. Our managers

now feel that we should have goals for all of our activities.

I am proud of the job that our management team has done in setting these new

goals. They provide a solid policy framework for planning and budgeting over the

next four years. I am also proud of the MBO planning and management system that

we have implemented to manage our activities. The Commerce systen is serving as a

model for several other agencies who are using it to implement similar s-ystems. The system has my full support and the support of my management team. We are

counting on each emiployee to continue supporting this system throughout the

planning period and to continue their excellence in achieving the new goals and

objectives. 4 

-

:;ecretary of Commerce

Attachment A-i

" " """'.'" ." .. " . . "" '. "". . . , . *"' ", , . , :" , " " '"'"""" ". "._ . ."," .
Major U.S. Department ,of Commerce Goals

Increase America's Competitiveness in the World Economy

- Increase U.S. exports of goods and services

- Expand foreign markets by removing barriers to travel, trade and investment

- Increase the number of small and medium sized exporters

- Increase U.S. share of the international travel market

- Increase exports and domestic consumption of U.S. fisheries products

- Increase the acceptance of international technical standards based on U.S. technology

- Strengthen worldwide protection of intellectual property

* Increase the competitiveness of U.S. telecommunications, computer and information

products and services

Safeguard the National Interest through Effective Administration of U.S. Trade

Laws

- Administer and enforce U.S. trade laws for reasons of national security, foreign

policy, and short supply

- Reduce export licensing time to an average of 30 days by 1990

* Expedite decontrol of non-critical commodities

- Take prompt, aggressive action against unfair trade practices

Stimulate Productivity and Economic Development

- Promote private sector capital formation, job creation, and new and expanding

• =industries

- Promote the development and application of science and technology in U.S.

industries

- Reduce regulatory and administrative barriers to innovation and growth of U.S.

business and industry

* Encourage the creation and expansion of minority businesses

. Reduce patent pendency to 18 months by 1987 and improve patent quality

- Promote the development and growth of oceanic and atmospheric industries

" Modernize domestic statutes to reflect the changing nature of the world econOmy.

A-2
Imp rove the Quality, Scope, Timeliness and Availability of Departmental Statistics

and Analyses

Develop complete plans for conducting the 1990 census

• Improve statistics on the.services-producing sector of the world economy

- Improve official economic and trade data and analyses

Manage Effectively the Nation's Oceanic and Atmospheric Resources

" Improve federal management of U.S. ocean and coastal resources

* Provide timely environmental data, information and assessments to U.S. industry

" Determine the characteristics and resources of the U.S. 200 mile exclusive

economic zone by 1990

" Increase our understanding of oceanic and atmospheric processes through

monitoring and research

Improve Service Delivery and Internal Management

* Provide accurate and timely information and services to U.S. travel and trade

exporters

* Provide more timely and accurate weather forecasts and warning services to the

U.S. public

* Restructure and modernize the national weather system by 1990

- Improve delivery of ocean information products and services

* Improve the Federal Government's use of computer and telecommunications

technology

* Automate the patent and trademark processes by 1990

- Make Departmental laboratories available for proprietary research

- Improve regional delivery of administrative services

- Manage Departmental functions to reduce competition with the private sector

and generate cost recovery from users

January 1985

A- 3

-... ..-. .... ....-.. , .,.. ..- . ... 
EXAMPLE OF STATEMENT OF TOP MANAGEMENT SUPPORT FOR TRAIt!IWG

DEPARTMENT OF THE ARMY

H_7.WADOUARTERS. U. S. ARMY MATtRIfL COMMAND

$001 JSCNOWCR AVCNUE. ALEXANORIA. VA 22333-OO",

AXCPE-CE-L 30 January 1985

SUBJECT: AMC Program on Improving Performance Standards

Commander

1. One of my primary concerns in the area of personnel management is the

quality of performance standards in AMC. Personnel Kanagement surveys by

OPH, BQDA, AMC and AMC HSC's continue to find that performance standards

need improvement. Additionally, inflated rating profiles indicate to me

that either performance standards are poor or evaluations are inaccurate

or both.

2. I am asking each AMC Commander and Activity Chief to take a personal

interest in improving performance standards vithin his/her command. In

support of this effort HQ AMC has developed jointly with OPH & course of

instruction on establishing individual performance plans. We have invited

each ANC activity with a CPO to name one personnelist and one other person,

preferably a line manager, to receive the basic instruction and to train

supervisors upon return to the inctallation/activity. Five workshops are

scheduled through February 1985 to train the 90 instructors.

3. The local training is designed to be given on three consecutive days.

We are not requiring that every civilian and military supervisor take this

training regardless of his/her expertise. However. we have seen no examples of standards to date which would indicate that a high level of

expertise exists. If it does exist, we would recommend that those who

possess it become a resource for the local training program at the earliest

point. We would expect most, if not all, supervisors of civilians to attend

this training.

4. Yf this training is attended and the principles covered are applied,

as am certain that betLer performance standards will reault and the investm ent of our supervisors' time will be well spent. Please give me your

support in this important effort.

Commanding

B-I

........................................ * **-.-. ...