United States General Accounting Office

GAO Report to Congressional Requesters

November 1995 FEDERAL JOB

CLASSIFICATION

Comparison of Job

Content With Grades

Assigned in Selected

Occupations

GAO/GGD-96-20

GAO United States

General Accounting Office

Washington, D.C. 20548

General Government Division

B-217675 

November 6, 1995

The Honorable John Glenn

Ranking Minority Member

Committee on Governmental Affairs

United States Senate

The Honorable William L. Clay

House of Representatives

The Honorable Eleanor Holmes Norton

House of Representatives

The Honorable Vic Fazio

House of Representatives

The Honorable Steny H. Hoyer

House of Representatives

Over the years, many studies have suggested that women and minorities

are paid less than men and nonminorities who work in comparable

positions. These observations have raised questions about whether the

federal government’s classification systems result in lower grades being

assigned to positions in occupations having large numbers of female or

minority incumbents than to other comparable occupations. This report

responds to your request that we determine whether the relationship

between job content and grades assigned using the Factor Evaluation

System (FES) varied on the basis of the proportions of women and

minorities in occupations.1

Background Position classification systems are formal methods for determining the

relative worth of positions in an organization. The Office of Personnel

Management (OPM) has responsibility and authority for federal position

classification, except for certain positions in agencies exempted by law.

OPM develops and issues classification standards and policies for the

federal personnel system. Federal agencies then use these standards and

policies to assign grades to positions.

The General Schedule (GS) is a 15-grade pay system that covers 442

white-collar occupations and approximately 1.5 million full-time

1

FES, which covers almost one-third of the federal full-time permanent white-collar workforce, is a

point-factor system for determining grades and is considered to be more orderly or rigorous than other

federal classification systems.

Page 1 GAO/GGD-96-20 Federal Job Classification
B-217675 

permanent employees in the federal white-collar workforce. Agencies

classify most of these positions using either narrative or FES point-factor

classification standards.2

 The FES primary standard serves as the

framework for individual classification standards written for each

occupation. When we began our study, FES standards were in effect for 77

occupations covering approximately 441,000 full-time permanent

nonsupervisory employees.

Under FES, positions are assigned grades on the basis of the duties,

responsibilities, and the qualifications required in terms of the following

nine factors:

• knowledge required by the position includes the skills needed to apply

that knowledge;

• supervisory controls entail the control exercised by the incumbent’s

supervisor (not the incumbent’s span of control over subordinates);

• guidelines provide reference data or impose certain constraints on the

incumbent’s use of knowledge; include desk manuals, established

procedures and policies, traditional practices, and materials such as

dictionaries, etc.; and vary by specificity, applicability, and availability;

• complexity consists of the nature, variety, and intricacy of tasks and the

difficulty and originality involved in performing the work;

• scope and effect encompass the relationship between the breadth and

depth of the work and the effect of work products and services within and

outside the organization;

• personal contacts refer to contacts with persons not in the supervisory

chain;

• purpose of contacts ranges from factual exchanges of information to

situations involving significant or controversial issues and differing

viewpoints, goals, or objectives;

• physical demands include physical characteristics such as agility,

dexterity, and physical exertion—such as climbing, lifting, pushing, etc.;

and

• work environment pertains to the risks and discomforts in the

employee’s physical surroundings or work assigned.

As shown in table 1, each factor is broken down into graduated levels.

Factors are composed of from three to nine levels with most having four

2

The point-factor system involves assigning a point value or weight to each compensable factor and

totaling the points assigned to obtain a job worth score that measures the relative importance of each

position to an organization. Narrative standards use fewer factors to describe the important

characteristics of the work, and the GS grade is determined through nonquantitative analysis.

Page 2 GAO/GGD-96-20 Federal Job Classification
B-217675 

to six levels. One factor—knowledge required by the position—has the

largest range of points, from 50 for level 1 to 1,850 for level 9.

Table 1: FES Factors and Points by

Factor Levels Levela

Factor 123456789

Knowledge required

by the position

50 200 350 550 750 950 1,250 1,550 1,850

Supervisory controls 25 125 275 450 650

Guidelines 25 125 275 450 650

Complexity 25 75 150 225 325 450

Scope and effect 25 75 150 225 325 450

Personal contacts 10 25 60 110

Purpose of contacts 20 50 120 220

Physical demands 5 20 50

Work environment 5 20 50

aThe blank spaces in columns indicate that no further levels exist for the corresponding factor.

Source: FES Primary Standard.

To determine a position’s GS grade, the agency typically compares either

the position description or information gathered through a “desk audit”

with the nine FES factors described in the classification standard.3

 After all

nine factors are evaluated, the points for all factors are totaled, and the

total for each position is converted to a GS grade by using a conversion

table (see table 2).

3

Desk audits typically consist of a trained classifier’s face-to-face interviews with the incumbent and

supervisor and observations of the work operations, sometimes including an examination of work

products.

Page 3 GAO/GGD-96-20 Federal Job Classification
B-217675 

Table 2: FES GS Grade Conversion

Table GS grade Point range

GS-1 190 - 250

GS-2 255 - 450

GS-3 455 - 650

GS-4 655 - 850

GS-5 855 - 1,100

GS-6 1,105 - 1,350

GS-7 1,355 - 1,600

GS-8 1,605 - 1,850

GS-9 1,855 - 2,100

GS-10 2,105 - 2,350

GS-11 2,355 - 2,750

GS-12 2,755 - 3,150

GS-13 3,155 - 3,600

GS-14 3,605 - 4,050

GS-15 4,055 - up

Source: FES Primary Standard.

Approach A contractor, with our supervision, developed a job content questionnaire

on the basis of the FES primary standard. The contractor distributed it to a

stratified random sample of 2,060 pairs of incumbents and their

supervisors and received responses from 1,639 incumbent/supervisor

pairs, which represents an overall response rate of about 80 percent.

Because individual federal positions are classified through labor-intensive

desk audits, it was not practical for us to study the majority of FES

occupations using traditional classification methods.

The contractor did, however, do desk audits of 78 judgmentally selected

positions and compared the audit results with the related questionnaire

responses. These comparisons indicated a fairly high correlation across

occupations. The validity coefficient between the GS grades resulting from

the desk audits and those from the questionnaires was .80 when the

incumbent and supervisor questionnaire responses were averaged.4

 We

considered this correlation to be sufficiently high to validate the use of the

4

A validity coefficient measures the consistency between two data sets.

Page 4 GAO/GGD-96-20 Federal Job Classification
B-217675 

job content questionnaire results for comparing groups of occupations.5

However, we cannot attest to the questionnaire’s validity when used

across GS grades within occupations, for specific occupations, or for

individual positions. Further, our study was not designed to determine

whether, and if so why, the job content reflected by the questionnaires

differed from that contained in the official job descriptions, which are the

bases for the actual GS grades.

We used an OPM database to identify a representative segment of

incumbents and their supervisors in occupations with high, medium, or

low representations of women or minorities. We defined occupations with

high, medium, and low female representation as those in which women

represented 70 percent or more, 31 to 69 percent, and 30 percent or less of

incumbents, respectively. We considered occupations in which minorities

represented more than 41 percent, 23 to 41 percent, or less than 23 percent

of incumbents as those with high, medium, and low minority

representation, respectively. Examples of occupations with high female

representation included secretary, dental hygienist, medical clerk, dental

assistant, and occupational therapist. Examples of occupations with high

minority representation included border patrol agent and computer and

equal employment opportunity specialists.

We selected our sample of incumbents and their supervisors from a total

of 58 occupations, which collectively represented about 90 percent of the

full-time permanent nonsupervisory employees covered by FES when we

designed our study. Because preliminary analyses indicated that more

variation in undergrading and overgrading existed among occupations

with similar gender and minority representation than between groups of

occupations with different gender and minority representations, we did

our analyses on the 37 occupations for which we had received completed

questionnaires from at least 10 or more incumbent/supervisor pairs, for a

total of 1,358 pairs or positions. These 37 occupations represented almost

one-quarter of the federal white-collar workforce, or about 79 percent of

the employees covered by FES, and the results of our study are

generalizable only to this population.

6

5

Psychological testing standards traditionally required validity coefficients of at least .70 for all tests,

and on that basis, we conclude that our result is sufficiently valid. However, psychologists have

acknowledged that coefficients as low as .30 can be of practical value. This is the only classification

study we are aware of in which desk audits were used to demonstrate that the questionnaire results

were similar to those which would be attained in an actual position classification.

6

Because we could not obtain reliable estimates of the odds of overgrading and undergrading for

occupations with small numbers of respondents, we deleted those 21 occupations with fewer than 10

incumbent/supervisor pairs.

Page 5 GAO/GGD-96-20 Federal Job Classification
B-217675 

On the basis of the average total factor level points derived from the

questionnaires completed by the incumbent and the supervisor, we

determined the GS grade for each position using the FES conversion table

(see table 2). We compared the questionnaire grade with the incumbent’s

actual GS grade and considered the position to be aptly, or appropriately,

graded if the questionnaire grade and the actual grade were the same.

Otherwise, we considered positions to be overgraded if the actual grade

was higher than the questionnaire grade and undergraded if the actual

grade was lower than the questionnaire grade.

Our study was not designed to permit us to

• approximate the number of positions that appeared to be overgraded,

undergraded, or aptly graded for the portion of the federal workforce to

which the results of our study are generalizable;

• identify the causes of any overgrading or undergrading resulting from

either (1) our use of the primary rather than the occupation-specific

classification standards, (2) the agencies’ application of classification

standards to individual positions, or (3) management decisions regarding

the work incumbents were actually assigned versus their job descriptions;

• determine whether any difference on the basis of gender or minority status

was inherent in the design of FES, as a product either of the factors that

constitute FES or the allocation of weight or the point range assigned to

each factor; or

• calculate what pay adjustments, if any, should be made.

Appendix I contains a more detailed discussion of the job content

questionnaire development and validation, sample selection, and response

rate calculation.

To determine the effects of female or minority representation on relative

overgrading and undergrading, we used odds and odds ratios. We

calculated the odds of occupations being undergraded rather than aptly

graded by dividing the number of positions undergraded by the number

aptly graded for groups of occupations. To determine how much more

likely one group of occupations was to be undergraded than another

group, we divided the odds of being undergraded for one group of

occupations by the odds for the other group to form an odds ratio. We

used the same procedures for overgrading.

We used loglinear analysis to determine how the odds of occupations

being overgraded or undergraded versus aptly graded varied (1) across the

Page 6 GAO/GGD-96-20 Federal Job Classification
B-217675 

range of GS grades and (2) when the female or minority representation of

the occupation was high, medium, or low. The strength of this particular

statistical approach is that multiple variables can be analyzed

simultaneously. Appendix II provides more detailed information on the

calculation of odds and odds ratios and loglinear models tested and the

results obtained.

To gain historical perspective, we reviewed previous studies of federal

classification issues. We also conferred with federal classification experts

to obtain additional insights about possible explanations for specific

findings.

This study should not be referred to as a “pay equity” study because we

examined only the relationship between job content and GS grades

assigned through the use of FES and whether that relationship varied with

the proportion of women or minorities in occupations.

We obtained comments from OPM that are discussed on pages 11 through

12 and presented in appendix III. We did our study from January 1990 to

September 1995 in accordance with generally accepted government

auditing standards.

Results in Brief For the nonsupervisory positions in the 37 occupations we studied, our

analyses suggest that the difference between the actual GS grades and

those we determined using a job content questionnaire was directly

related to the female and minority representation of the occupations we

examined.7

 If the actual grade was higher than the questionnaire-derived

grade, we considered the position to be overgraded; if the actual grade

was lower than the questionnaire grade, we considered the position to be

undergraded.

The likelihood of a position being overgraded, rather than aptly graded,

increased as the incumbents’ GS grades increased. However, the

incumbents’ grades had virtually no effect on the likelihood that a position

was undergraded versus aptly graded. After statistically eliminating this

effect of the incumbents’ GS grades, we found that the occupations we

7

We developed our job content questionnaire on the basis of the FES primary standard that serves as

the framework for individual classification standards written for each occupation. The actual GS

grades were assigned using these occupation-specific standards. Further, the questionnaire was

designed and validated to achieve our review objectives relative to comparing groups of occupations.

It was not designed as and is not a valid substitute for traditional classification methods. Therefore,

the questionnaire should not be used to draw definitive conclusions regarding overgrading or

undergrading of individual positions.

Page 7 GAO/GGD-96-20 Federal Job Classification
B-217675 

studied with high female representation were more likely to be

undergraded rather than aptly graded compared with the occupations

having medium or low female representation. As mentioned above, we

defined occupations with high female representation as those in which

70 percent or more of the incumbents were women. This definition was

used on the basis of the current literature when we designed our study.

The literature contained no consistent basis on which to define minority

representation. On the basis of our past work, we initially adopted a

standard of more than 48 percent to define high minority representation.8

However, we found it necessary to adjust our definition for high minority

representation to occupations in which more than 41 percent of the

incumbents were minorities in order to include at least two occupations

with each possible mix of gender and minority representation (e.g., low

female, high minority representation).

We would have preferred that our original definition had provided a

sufficient mix of occupations in our sample. And it is important to note

that about half of the incumbents in the total population of the

occupations defined by our study as having a high minority representation

were in fact nonminorities. Within these parameters, after eliminating the

effects of the incumbents’ GS grades, we found that occupations with high

minority representation were more likely to be overgraded rather than

aptly graded in comparison with the occupations having medium or low

minority representation.

The National Performance Review has contended that the current federal

classification systems have too many occupations and grades and has

recommended that a more flexible “broad-banded” system be adopted. OPM

is currently, within the existing statutory framework, planning to revise

the classification standards and increase classification oversight. Our

study suggests any new system should be closely monitored to ensure that

unintended disparities are identified and addressed.

8

When we designed our study, minorities comprised approximately 32 percent of the workforce

covered by FES. We initially defined occupations with high minority representation as those in which

more than 48 percent (150 percent of the 32 percent minority workforce representation) of the

incumbents were minorities.

Page 8 GAO/GGD-96-20 Federal Job Classification
B-217675 

Effects of Female and

Minority

Representation on

Relative Overgrading

and Undergrading

To evaluate whether gender or minority representation had an effect on

the variation between the actual grades and the questionnaire grades, we

statistically eliminated the effect of the incumbents’ GS grades.9

 Analysis of

the remaining variation showed that occupations with high female

representation were 1.77 times more likely to be undergraded rather than

aptly graded compared with occupations having a low or medium female

representation. Occupations with high minority representation were 2.18

times more likely to be overgraded rather than aptly graded compared

with occupations having a low or medium minority representation.10

Classification experts with whom we consulted about our results and the

available literature offered a few occupation-specific hypotheses about

possible causes. For example, key occupations with high minority

representation that appeared to be overgraded included (1) border patrol

agents, (2) equal employment opportunity and compliance specialists, and

(3) computer specialists. Previous studies of federal classification issues

maintained that FES was ineffective for specialists such as law

enforcement related occupations because physical demands and work

environment are not highly valued FES factors but are considered

significant in these occupations.

Although empirical data are lacking, the classification experts we

consulted suggested that when equal employment opportunity

occupations were established in the 1970s, they involved a heavy workload

of cases and, even though not recognized by FES, the GS grades of these

occupations may have been increased on that basis. Furthermore, private

sector wages may have resulted in overgrading positions in

computer-related occupations. Explanations are somewhat less evident

regarding occupations with high female representation, which appear

more likely to be undergraded.

9

The likelihood or odds of a position being overgraded increased as the GS grade increased; that is, as

the grade increased by one GS grade, the odds that a position was overgraded versus aptly graded

increased by a factor of 1.64. For example, the odds of positions being overgraded in occupations with

an average grade of GS-9 were approximately 5 to 1 (i.e., five positions were overgraded for every one

that was aptly graded), while the odds of positions being overgraded in occupations with an average

grade of GS-10 were about 8.2 to 1. The factor of 1.64 (8.2 / 5 = 1.64) indicates that positions in the

higher graded occupations were 1.64 times more likely than those in the lower graded occupations to

be overgraded. At the 95 percent level of confidence, as the grade increased by one GS grade, the odds

that a position was overgraded versus aptly graded increased by a factor ranging from 1.53 to 1.77.

The GS grade had no significant effect on the odds that a position was undergraded versus aptly

graded.

10At the 95 percent level of confidence, occupations with high female representation were from 1.33 to

2.35 times more likely to be undergraded than occupations with medium or low female representation,

and occupations with high minority representation were from 1.62 to 2.94 times more likely to be

overgraded than those with medium or low representation.

Page 9 GAO/GGD-96-20 Federal Job Classification
B-217675 

Current Efforts to

Revamp Classification

Standards and

Systems

Critics of the federal classification system, including reports from the

National Academy of Public Administration and the National Performance

Review, have argued that the classification system is overly complex, with

too many occupations and GS grades, and that a less rigid system would be

more effective. They have recommended fewer occupations and more

flexible broad-banded grade structures.

OPM officials told us recently that, within the existing statutory framework,

OPM plans to revise the classification standards and increase its oversight

of various processes including classification. A current proposal for

rewriting classification standards would reduce the inventory of 442

white-collar classification standards to about 74. Rather than individual

occupations, the new standards would focus on the 22 “job families” of

related occupations with separate standards, as applicable, for

professional, administrative, technical, and clerical positions.11

OPM has also established a new oversight office, which, among other

things, is planning various governmentwide policy studies. OPM has

tentatively allocated about 145 staff years to this effort; most of these

resources are located in field offices rather than at headquarters. One of

the highest priorities will be a governmentwide classification study, with

particular emphasis on determining the accuracy of “border

grades”—those grades most likely to be placed at the lower and upper

limits of any newly created grade bands. A team is examining options for

doing this study, and work on the study is scheduled to begin early in

fiscal year 1996.

Conclusions Although FES is considered a more orderly or rigorous method than other

federal classification systems, our study identified differences in the

grading of positions in occupations with high representations of women or

minorities. The National Performance Review and other studies suggest

that the current classification systems should be abandoned in favor of

more flexible, broad-banded systems. The results of our study indicate that

it is important that policymakers closely monitor any new systems to

ensure that (1) unintended disparities are identified so that they can be

corrected and (2) the national policy underlying the current classification

system—that jobs be classified so that pay is equal for substantially equal

work—is being satisfactorily achieved. Since OPM is in the process of

11“Job families” consist of related occupations that are grouped together such as the General

Administrative, Clerical, and Office Service Group; the Accounting and Budget Group; the Physical

Science Group; etc.

Page 10 GAO/GGD-96-20 Federal Job Classification
B-217675 

substantially revising the classification and oversight systems, we are

making no recommendations in this report.

Agency Comments

and Our Evaluation

OPM provided written comments on a draft of this report. (See app. III.)

OPM took issue with our methodology, saying that it was insufficient to

support our findings. OPM also discussed its plans to further explore job

classification accuracy issues.

OPM took exception with our methodology in two respects. First, it

believes we inappropriately used the primary classification standard

rather than occupation-specific standards as the basis for our job content

questionnaire. OPM said that the primary standard was designed to be used

as an overall outline wherein more specific standards would be developed,

not as a basis for evaluating individual positions. Although we used the

primary standard rather than the occupation-specific standards in the

development of our methodology, we examined the specific standard for

several occupations in our sample to see if we could identify any ways in

which the use of the occupation-specific standard might have led to a

different result. We did not identify any such effect. We also asked OPM’s

classification experts to identify occupations in our sample for which the

specific standards were, in their view, sufficiently different from the

primary standard that our results would have been affected in identifiable

ways. They did not identify any such occupations. Thus we continue to

believe that our use of the primary standard was appropriate and that our

methodology produced useful results.

OPM’s second exception with our methodology was our use of a job

content questionnaire rather than traditional desk audits to assign grades

to positions. OPM said that our use of a questionnaire resulted in employees

and supervisors, unfamiliar with FES ground rules, being asked to select

generic phrases that were not in context and that this in turn resulted in

the grades we assigned being less credible than those derived by federal

agencies. We acknowledge in our text that actual GS grades are assigned

on the basis of occupation-specific standards and that the desk audit is a

typical way to assign a grade to a position. As noted in the text, desk

audits are labor-intensive, and it was not practical for us to study the

majority of FES occupations using traditional classification methods.

Because of this, we took care to validate our results. First, we had a

contractor do desk audits on positions in a number of occupations in our

sample. Next, we compared the results of those desk audits with the

questionnaire results for those positions. This comparison showed a fairly

Page 11 GAO/GGD-96-20 Federal Job Classification
B-217675 

high correlation between the GS grades resulting from desk audits and

those from the questionnaire. Thus, we believe that our methodology is

appropriate to identify patterns of overgrading and undergrading among

groups of occupations with different representations of women or

minorities.

OPM also questioned our study results by comparing them with other

studies. More specifically, OPM said that other agencies’ studies and OPM’s

classification appeals data indicated lower levels of misclassification than

our study. We are unaware of any recent studies or appeals data in which a

direct comparison with our study could be meaningful. Although OPM’s

most recent report on the overall federal white-collar position

classification accuracy indicated a lower level of misclassification than

our study, it was published in 1983. We acknowledge that classification

appeals also indicate a lower level of misclassification than our study.

However, classification experts with whom we consulted said that appeals

data are unlikely to represent those federal employees whose positions

may be overgraded. As indicated by the Merit System Protection Board

(MSPB), almost no one files a classification appeal.12

Finally, OPM said that it shares with us the need to ensure that the federal

government’s classification systems and their applications are fair and

unbiased. OPM said that to this end, its newly designed oversight program

will have a major focus on ensuring that current and new classifications

systems advance the merit principles of equal pay and the efficient and

effective use of the federal workforce. OPM said that it expects to decide on

the classification review design by the end of fiscal year 1995 and begin

work on the review in early fiscal year 1996.

We are sending copies of this report to interested Members of Congress

and congressional committees that have responsibilities for public sector

employment issues, the Director of the Office of Personnel Management,

and other interested parties. Copies will also be made available to others

upon request.

12According to MSPB only 240 of the government’s 2.1 million civilian nonpostal employees filed an

appeal in 1988, or about one one-hundredth of 1 percent.

Page 12 GAO/GGD-96-20 Federal Job Classification
B-217675 

Major contributors to this report are listed in appendix IV. If you have any

questions about the report, please call me at (202) 512-7824.

Nancy Kingsbury

Director

Planning and Reporting

Page 13 GAO/GGD-96-20 Federal Job Classification
Contents

Letter 1

Appendix I 

Questionnaire

Development, Sample

Selection, Response

Rate Calculation, and

Validation

Methodology

18

Questionnaire Development and GS Grade Estimation 19

Sample Selection 24

Response Rate Calculation 33

Occupations Included in the Study Analyses 34

Questionnaire Validation 35

Appendix II 

Loglinear

Methodology and

Analysis Results

39

Data Analysis Approach 39

Occupations Included in Our Study 39

Relationship Between Overgrading/Undergrading and Female or

Minority Representation Without Controlling for GS Grade

45

Relationship Between Overgrading/Undergrading Versus GS

Grade and Female and Minority Representation

50

Appendix III 

Comments From the

Office of Personnel

Management

63

Appendix IV 

Major Contributors to

This Report

68

Bibliography 69

Related GAO Products 144

Tables Table 1: FES Factors and Points by Factor Levels 3

Table 2: FES GS Grade Conversion Table 4

Table I.1: FES Factors and Points by Factor Levels 20

Table I.2: Conversion of FES Points to GS Grade 23

Page 14 GAO/GGD-96-20 Federal Job Classification
Contents

Table I.3: Example of GS Grade Estimation Based on Incumbent

and Supervisor Questionnaire Responses

24

Table I.4: Distribution of the 77 FES Occupations 25

Table I.5: Female and Minority Representation, PATCO Category,

and Number of Incumbents for the 77 Occupations Covered by

FES

27

Table I.6: Distribution of the 58 Occupations Included in the

Sample

30

Table I.7: Distribution of the 2,233 Incumbents by Female and

Minority Representation and Strata

32

Table I.8: Disposition of Incumbent/Supervisor Pairs in the

Sample

33

Table I.9: Female and Minority Representation of the 37

Occupations Included in the Study

35

Table I.10: Distribution of Desk Audits Completed According to

Female and Minority Representation for the 37 Occupations

Included in the Study

36

Table I.11: Validity Coefficients for Grades Estimated on the

Basis of the 78 Desk Audits Versus the Questionnaire Responses

for Incumbent/Supervisor Pairs, Supervisors Only, and

Incumbents Only

38

Table II.1: Female and Minority Representation, Average GS

Grades, Observed Sample Frequencies, and Odds of Being

Overgraded and Undergraded Versus Aptly Graded for 37

Occupations Without Controlling for GS Grade

42

Table II.2: Relationship Between the Observed Sample

Frequencies and Female Representation Without Controlling for

GS Grade

46

Table II.3: Hierarchical Models Tested to Examine Relationships

Between Observed Sample Frequencies and Female and Minority

Representation

47

Table II.4: Relationship Between the Expected Frequencies and

Female Representation Without Controlling for GS Grade

48

Table II.5: Relationship Between the Observed Sample

Frequencies and Minority Representation Without Controlling for

GS Grade

49

Table II.6: Relationship Between the Expected Frequencies and

Minority Representation Without Controlling for GS Grade

50

Table II.7: Loglinear Models Tested to Examine the Relationship

Between the Frequencies, the GS Grade, and Female and

Minority Representation

54

Page 15 GAO/GGD-96-20 Federal Job Classification
Contents

Table II.8: Female and Minority Representation, Average GS

Grades, Expected Frequencies, and Odds of Overgraded and

Undergraded Versus Aptly Graded for 37 Occupations

58

Table II.9: Coefficients and Odds Ratios Describing the Effects of

the Average GS Grade and High Female and Minority

Representation on Overgrading and Undergrading

62

Figures Figure I.1: Excerpt from the FES Primary Standard 21

Figure I.2: Excerpt from the Job Content Questionnaire 22

Figure II.1: Odds of Overgrading by Average GS Grade Across 37

Occupations

51

Figure II.2: Odds of Undergrading by Average GS Grade Across 37

Occupations

52

Abbreviations

CPDF Central Personnel Data File

FES Factor Evaluation System

GS General Schedule

MSPB Merit System Protection Board

OPM Office of Personnel Management

PATCO Professional, Administrative, Technical, Clerical, Other

Page 16 GAO/GGD-96-20 Federal Job Classification
Page 17 GAO/GGD-96-20 Federal Job Classification
Appendix I 

Questionnaire Development, Sample

Selection, Response Rate Calculation, and

Validation Methodology

Using the competitive bid process, we contracted with a national

management consultant—Barrett & Associates, Inc., of Akron, Ohio—to

(1) develop a job content questionnaire that enabled the contractor to

estimate GS grades for positions in our nationwide sample using input from

the incumbents and their supervisors, (2) distribute the questionnaire to a

sample of incumbent/supervisor pairs attaining at least an 80 percent

response rate, (3) develop a semistructured interview guide for completing

desk audits, (4) validate the questionnaire through the use of desk audits,

and (5) analyze the questionnaire responses.1

 The analyses were done to

determine whether a link existed between the GS grades assigned to

positions through the use of the Factor Evaluation System (FES) and the

number of women or minorities in occupations that we included in our

study.

We selected the sample of incumbent/supervisor pairs and worked closely

with the contractor providing supervision throughout the process.

Because the preliminary analyses showed that more variation existed

within groups of occupations rather than between those groups, we

refined the analyses of the questionnaire responses as described in

appendix II.

In addition to working with the contractor, we consulted periodically with

an ad hoc panel of experts that provided technical guidance during the

early design phase of our study. Our panel consisted of

• Ms. Ruth Rogers, former Chief, Standards Division, Department of

Personnel, Government of the District of Columbia, who provided staff

assistance to the District’s pay equity study;2

• Dr. Donald Schwab, Professor of Business Research and Industrial

Relations, University of Wisconsin-Madison, who has written extensively

on the issue of comparable worth; and

• Dr. Ronnie Steinberg, Professor of Sociology and Women’s Studies,

Temple University, who has considerable experience with pay equity

issues and job content questionnaires.

1

Desk audits consist of face-to-face interviews between trained classifiers and the incumbent and

supervisor to identify and observe work operations and products. The validation was done to

determine whether our questionnaire could be used to estimate GS grades similar to those that would

be attained in actual desk audits. When we designed our study, we set a goal of achieving a .80 validity

coefficient, which measures the consistency between two data sets.

2

According to a District of Columbia official, in July 1990, the District contracted with a consulting firm

to do a pay equity study, and in April 1994, the contractor provided a final report with

recommendations.

Page 18 GAO/GGD-96-20 Federal Job Classification
Appendix I 

Questionnaire Development, Sample

Selection, Response Rate Calculation, and

Validation Methodology

We also consulted with Dr. David Rindskopf, Professor, Graduate School

and University Center, City University of New York, for assistance with

our sample selection methodology.

After we completed our analyses, we shared our results with and obtained

informal comments from five federal officials with classification

experience. In addition to a representative from OPM and GAO’s personnel

office, we selected three of these experts from a list of recommended

candidates provided at our request by the Classification and

Compensation Society, a professional organization of federal classifiers

and other personnel specialists. The classification experts included a mix

of male, female, minority, and nonminority individuals who worked in

OPM’s then Personnel Systems and Oversight Group and in the personnel

offices of the Federal Aviation Administration, the Department of Defense,

the Library of Congress, and the General Accounting Office.

Questionnaire

Development and GS

Grade Estimation

We developed a job content questionnaire that enabled us to estimate GS

grades for positions covered by FES, a point factor position classification

system the federal government used to classify 77 of the 455 white-collar

occupations when we designed our study.3

 On the basis of the factor

descriptions in the FES primary standard, we constructed questions that

allowed us to determine the appropriate level for each factor.4

 Table I.1

shows the points associated with each factor level.

3

A point factor position classification system uses a set of factors and factor weights to order positions

hierarchically in terms of their value to an employer. FES uses the following nine factors to describe

the duties and responsibilities of a position: knowledge required by the position, supervisory controls,

guidelines, complexity, scope and effect, personal contacts, purpose of contacts, physical demands,

and work environment. Each factor is broken down into graduated levels. Factors are composed of

from three to nine levels.

4

The FES primary standard serves as the framework for individual classification standards written for

each occupation. We compared the occupation-specific standards with the primary standard for four

occupational series—computer operations, border patrol agent, equal employment opportunity, and

equal opportunity compliance—and determined that these occupation-specific standards appeared to

be within the framework of the primary standard.

Page 19 GAO/GGD-96-20 Federal Job Classification
Appendix I 

Questionnaire Development, Sample

Selection, Response Rate Calculation, and

Validation Methodology

Table I.1: FES Factors and Points by

Factor Levels Levela

Factor 123456789

Knowledge required

by the position

50 200 350 550 750 950 1,250 1,550 1,850

Supervisory controls 25 125 275 450 650

Guidelines 25 125 275 450 650

Complexity 25 75 150 225 325 450

Scope and effect 25 75 150 225 325 450

Personal contacts 10 25 60 110

Purpose of contacts 20 50 120 220

Physical demands 5 20 50

Work environment 5 20 50

aThe blank spaces in columns indicate that no further levels exist for the corresponding factor.

Source: FES Primary Standard.

Figures I.1 and I.2 show an excerpt from the FES primary standard and the

resulting question that we included in our job content questionnaire,

respectively.

Page 20 GAO/GGD-96-20 Federal Job Classification
Appendix I 

Questionnaire Development, Sample

Selection, Response Rate Calculation, and

Validation Methodology

Figure I.1: Excerpt From the FES

Primary Standard

 Factor 1 measures the nature and extent of information or facts which 

the workers must understand to do acceptable work (e.g., steps, 

procedures, practices, rules, policies, theories, principles, and concepts) 

and the nature and extent of the skills needed to apply those knowledges. 

To be used as a basis for selecting a level under this factor, a knowledge 

must be required and applied.


Level 1-1

 Knowledge of simple, routine, or repetitive tasks or operations which 

typically includes following step-by-step instructions and requires little or 

no previous training or experience;


Level 1-2

 Knowledge of basic or commonly-used rules, procedures, or operations 

which typically requires some previous training or experience;


50 points

200 points

Level 1-8

 Mastery of a professional or administrative field to:


 


 – Apply experimental theories and new developments to problems not 

susceptible to treatment by accepted methods;


Level 1-9

 Mastery of a professional field to generate and develop new hypotheses and theories;

1550 points

FACTOR 1. KNOWLEDGE REQUIRED BY THE POSITION

1850 points

Page 21 GAO/GGD-96-20 Federal Job Classification
Appendix I 

Questionnaire Development, Sample

Selection, Response Rate Calculation, and

Validation Methodology

Figure I.2: Excerpt From the Job

Content Questionnaire

Tasks or operations involving specific step-bystep instructions requiring little or no previous 

training or experience


A few basic or commonly used rules, procedures, 

or operations plus some previous or on-the-job 

training or experience

13. To perform your job in a satisfactory manner, what level of 


knowledge and skill is needed? (Check one.)

1.

2.

Mastery of a professional, technical or 

administrative field to the point where one can 

apply new hypotheses, theories, or applications to 

applied problems


Mastery of a professional, technical or 

administrative field to the point where one can 

generate and develop new hypotheses and 

theories

8.

9.

The GS grade is determined by totaling the points assigned to each of the

nine factors and using the grade conversion table shown in table I.2.

Page 22 GAO/GGD-96-20 Federal Job Classification
Appendix I 

Questionnaire Development, Sample

Selection, Response Rate Calculation, and

Validation Methodology

Table I.2: Conversion of FES Points to

GS Grade GS grade Point range

GS-1 190 - 250

GS-2 255 - 450

GS-3 455 - 650

GS-4 655 - 850

GS-5 855 - 1,100

GS-6 1,105 - 1,350

GS-7 1,355 - 1,600

GS-8 1,605 - 1,850

GS-9 1,855 - 2,100

GS-10 2,105 - 2,350

GS-11 2,355 - 2,750

GS-12 2,755 - 3,150

GS-13 3,155 - 3,600

GS-14 3,605 - 4,050

GS-15 4,055 - up

Source: FES Primary Standard.

To determine whether incumbents could readily understand the

questionnaire and complete it within a reasonable time period, we

completed three sets of pretests. In total, we selected about 30 incumbents

and supervisors who were located in the Cleveland metropolitan area and

who represented a range of GS grades and occupations covered by FES.

After observing them as they completed the questionnaire, we interviewed

each incumbent or supervisor to identify how the questionnaire could be

improved; we rewrote or edited most of the questions on the basis of the

information they provided. During the initial pretest, we also revised our

semistructured interview guide for use in completing subsequent desk

audits.

On the basis of questionnaire responses received from incumbents and

their supervisors, we estimated the appropriate GS grade for each position

in our sample by (1) determining the appropriate level (and corresponding

points) for each of the nine factors, (2) averaging the total points

computed separately for the incumbent and the supervisor, and (3) using

the FES points-to-grade conversion table to assign a GS grade to the

position. We decided to average the input from the incumbent and the

supervisor to balance the views of those who would place more reliance

Page 23 GAO/GGD-96-20 Federal Job Classification
Appendix I 

Questionnaire Development, Sample

Selection, Response Rate Calculation, and

Validation Methodology

on the input of incumbents versus that of supervisors. Table I.3 shows an

example of how we estimated the GS grade for one position in our sample.

Table I.3: Example of GS Grade

Estimation Based on Incumbent and

Supervisor Questionnaire Responses

Incumbent Supervisor

Factor

Factor

level Points

Factor

level Points

Average

total

points

Knowledge required by

the position

4 550 3 350

Supervisory controls 2 125 2 125

Guidelines 2 125 1 25

Complexity 3 150 1 25

Scope and effect 2 75 4 225

Personal contacts 3 60 1 10

Purpose of contacts 1 20 1 20

Physical demands 2 20 1 5

Work environment 1 5 2 20

Total points 1,130 805 967

Estimated GS grade GS-5

Note: The blank spaces in columns indicate that we estimated GS grades by averaging the total

points on the basis of input from the incumbent and the supervisor rather than the points

assigned to each factor.

Sample Selection We selected our nationwide sample from the full-time permanent

nonsupervisory incumbents in the 77 occupations for which an FES

standard had been in existence for at least 1 year when we completed our

study design in May 1992. Because of limited resources, we excluded

incumbents who (1) were stationed outside of the continental United

States or (2) worked for agencies with less than 500 employees. Each of

the 77 occupations included from 35 to 91,769 incumbents for a total of

441,189 full-time permanent nonsupervisory incumbents. Women

constituted about 61 percent of the total workforce covered by FES and

minorities, approximately 32 percent. We used OPM’s Central Personnel

Data File (CPDF), updated as of June 1992, to select our sample. These

were the most recent data available when we designed our study. We did

not independently verify the accuracy of this database.

We defined each of the 77 occupations covered by FES as having a high,

medium, or low representation of women or minorities. On the basis of the

current literature, we defined occupations with high, medium, and low

Page 24 GAO/GGD-96-20 Federal Job Classification
Appendix I 

Questionnaire Development, Sample

Selection, Response Rate Calculation, and

Validation Methodology

female representation as those in which women represented 70 percent or

more, 31 to 69 percent, and 30 percent or less of the incumbents,

respectively. The literature contained no consistent basis on which to

define minority representation; therefore, on the basis of our past work,

we initially adopted a standard of more than 48 percent (150 percent of the

32 percent minority workforce representation) to define high minority

representation; less than 16 percent (50 percent of 32 percent), low

minority representation; and 16 to 48 percent, medium minority

representation. However, in order to include in our sample at least two

occupations with each possible mix of gender and minority representation

(e.g., low female, high minority representation), we found it necessary to

adjust our definitions for occupations with high, medium, and low

minority representation to those in which minorities represented more

than 41 percent, 23 to 41 percent, and less than 23 percent of incumbents,

respectively. Table I.4 shows the distribution of the 77 occupations in a

nine-cell matrix configured according to female and minority

representation.

Table I.4: Distribution of the 77 FES

Occupations Minority representation

Female representation High Medium Low Total

High 6 14 2 22

Medium 12 10 5 27

Low 3 3 22 28

Total 21 27 29 77

Source: GAO analysis of data obtained from OPM.

From the 77 occupations covered by FES, we selected 58 on the basis of

four characteristics—number of incumbents, PATCO (professional,

Page 25 GAO/GGD-96-20 Federal Job Classification
Appendix I 

Questionnaire Development, Sample

Selection, Response Rate Calculation, and

Validation Methodology

administrative, technical, clerical, and other) category,5 job family,6 and GS

grade distribution. First, we chose occupations in each matrix cell with the

largest number of incumbents. Second, we examined the PATCO category of

the occupations selected within each cell and chose additional

occupations to include all categories. Third, we reviewed the job families

represented by the occupations already selected and chose additional

occupations increasing the number of families included in our study to 17

out of a total of 22. Finally, we examined the GS grade distributions of the

occupations selected and chose occupations that included grades not

previously selected within each row and column of the matrix. Table I.5

shows the female and minority representation, PATCO category, and

number of full-time permanent nonsupervisory incumbents for each of the

77 occupations covered by FES when we selected our sample as well as the

58 occupations selected for inclusion in our sample.

5

OPM assigns a PATCO category to each white-collar occupation on the basis of the general subject

matter of work, level of difficulty or responsibility, and educational requirements.

Professional occupations require incumbents to use discretion and judgment to apply knowledge

acquired through education or training equivalent to a bachelor’s degree in a specialized field.

Administrative occupations involve the exercise of analytical ability and personal responsibility to

apply concepts and practices—typically learned through a general college education or progressively

responsible work—to one or more fields of management.

Technical occupations consist of nonroutine work that is learned on-the-job or from specialized

training less than that represented by college graduation to support professional or administrative

fields.

Clerical occupations require incumbents to do structured work according to established policies,

which are learned through training or work experience, to support office operations.

Other occupations include those miscellaneous occupations that are not included in one of the four

other categories.

6

Job families consist of related occupations that are numbered using the same multiple of 100. For

example, GS-300, the general administration, clerical, and office services group includes the mail and

file (GS-305), computer operation (GS-332), and telecommunications (GS-391) occupations.

Page 26 GAO/GGD-96-20 Federal Job Classification
Appendix I 

Questionnaire Development, Sample

Selection, Response Rate Calculation, and

Validation Methodology

Table I.5: Female and Minority Representation, PATCO Category, and Number of Incumbents for the 77 Occupations

Covered by FES

Representation Occupational

seriesa Female Minority PATCOb

Full-time permanent

nonsupervisory

incumbents

318 Secretary High Medium C 91,769

334 Computer specialist Medium Medium A 43,958

1102 Contracting Medium Medium P 25,355

525 Accounting technician High Medium C,Tc 18,081

621 Nursing assistant Medium High T 14,984

305 Mail and file Medium High C 14,630

592 Tax examining High Medium C,Td 13,271

1910 Quality assurance Low Low A 12,163

203 Personnel clerical and assistance High High C,Td 11,640

511 Auditing Medium Low P 11,413

560 Budget analysis High Medium A 11,099

830 Mechanical engineering Low Low P 10,726

679 Medical clerk High High C 10,697

510 Accounting Medium Medium P 8,838

335 Computer clerk and assistant High Medium C,Te 8,353

332 Computer operation Medium High T 6,833

610 Nurse High Medium P 5,746

1320 Chemistry Low Low P 5,347

83 Police Low Medium O 5,096

80 Security administration Medium Medium A 4,616

391 Telecommunications Low Low A 4,568

644 Medical technologist Medium Medium P 4,491

457 Soil conservation Low Low P 4,027

540 Voucher examining High Medium C 3,978

(561) (Budget clerical and assistance) (High) (Medium) (C,Td) (3,937)

85 Security guard Low High O 3,882

(1370) (Cartography) (Low) (Low) (P) (3,817)

460 Forestry Low Low P 3,764

(544) (Civilian pay) (High) (Medium) (C,Te) (3,524)

(819) (Environmental engineering) (Low) (Low) (P) (3,461)

950 Paralegal specialist High Medium A 3,460

18 Safety and occupational health

management

Low Low A 3,420

1035 Public affairs Medium Low A 3,202

1896 Border patrol agent Low High O 3,176

(continued)

Page 27 GAO/GGD-96-20 Federal Job Classification
Appendix I 

Questionnaire Development, Sample

Selection, Response Rate Calculation, and

Validation Methodology

Representation Occupational

seriesa Female Minority PATCOb

Full-time permanent

nonsupervisory

incumbents

475 Agricultural management Low Low P 3,088

1311 Physical science technician Low Low T 2,987

(545) (Military pay) (High) (Medium) (C,Te) (2,935)

(840) (Nuclear engineering) (Low) (Low) (P) (2,873)

681 Dental assistant High High T 2,660

620 Practical nurse High High T 2,563

647 Diagnostic radiologic technologist Medium Medium T 2,433

2134 Shipment clerical and assistance High Medium C 2,070

622 Medical supply aide and technician Medium High T 1,986

260 Equal employment opportunity Medium High A 1,907

(486) (Wildlife biology) (Low) (Low) (P) (1,850)

1173 Housing management Medium Medium A 1,841

(458) (Soil conservation technician) (Medium) (Low) (T) (1,835)

649 Medical instrument technician Medium High T 1,819

350 Equipment operator Medium High C 1,779

808 Architecture Low Medium P 1,686

360 Equal opportunity compliance Medium High A 1,526

(1980) (Agricultural commodity grading) (Low) (Low) (T) (1,438)

1371 Cartographic technician Medium Low T 1,358

(482) (Fishery biology) (Low) (Low) (P) (1,358)

690 Industrial hygiene Medium Low P 1,247

1822 Mine safety and health Low Low T 1,190

1889 Import specialist Medium Medium A 1,159

630 Dietician and nutritionist High Medium P 1,111

436 Plant protection and quarantine Low Medium P 1,097

701 Veterinary medical science Low Low P 834

188 Recreation specialist Medium Medium A 805

1040 Language specialist Medium High A 585

(651) (Respiratory therapist) (Medium) (High) (T) (496)

361 Equal opportunity assistance High High T 486

270 Federal retirement benefits Medium Medium A 416

1720 Education program Medium High P 369

(485) (Wildlife refuge management) (Low) (Low) (P) (365)

682 Dental hygiene High Low T 335

(2121) (Railroad safety) (Low) (Low) (A) (322)

309 Correspondence clerk High High C 310

(2123) (Motor carrier safety) (Low) (Low) (A) (284)

(continued)

Page 28 GAO/GGD-96-20 Federal Job Classification
Appendix I 

Questionnaire Development, Sample

Selection, Response Rate Calculation, and

Validation Methodology

Representation Occupational

seriesa Female Minority PATCOb

Full-time permanent

nonsupervisory

incumbents

(669) (Medical records administration) (High) (Medium) (A) (170)

(1161) (Crop insurance administration) (Low) (Low) (A) (109)

(1162) (Crop insurance underwriting) (Low) (Low) (A) (54)

631 Occupational therapist High Low P 49

(1884) (Customs patrol officer) (Low) (High) (O) (47)

(160) (Civil rights analysis) (Medium) (High) (A) (35)

Subtotal/included

in study

412,279

Subtotal/not

included in study

(28,910)

Total 441,189

Note: This table includes occupational series, or occupations, covered by FES that had been in

existence for at least 1 year as of May 1992.

aWe selected incumbents for our sample from the 58 occupational series not shown in

parentheses.

bThe PATCO category indicates the general characteristics of the work done within each federal

white-collar occupation and is represented as Professional, Administrative, Technical, Clerical, or

Other.

cThe PATCO category for this occupation is Clerical for GS grades 1 to 3 and Technical for

grades 4 and above.

dThe PATCO category for this occupation is Clerical for GS grades 1 to 5 and Technical for

grades 6 and above.

eThe PATCO category for this occupation is Clerical for GS grades 1 to 4 and Technical for

grades 5 and above.

Source: GAO analysis of data obtained from OPM.

We selected no more than 11 occupations for each matrix cell because we

planned to validate the job content questionnaire by completing two sets

of 100 desk audits that would each (1) include at least one position from

each occupation in our sample and (2) be evenly distributed among the 9

matrix cells. Table I.6 shows the distribution of the 58 occupations

selected for inclusion in our sample by matrix cell.

Page 29 GAO/GGD-96-20 Federal Job Classification
Appendix I 

Questionnaire Development, Sample

Selection, Response Rate Calculation, and

Validation Methodology

Table I.6: Distribution of the 58

Occupations Included in the Sample Minority representation

Female representation High Medium Low Total

High 6 10 2 18

Medium 10 10 4 24

Low 2 3 11 16

Total 18 23 17 58

Source: GAO analysis of data obtained from OPM.

After selecting the 58 occupations, we completed a pilot test by

distributing the job content questionnaire to 389 pairs of incumbents and

their supervisors who were located in either the Cleveland, OH, or

Washington, D.C. areas and who represented a range of GS grades and

occupations included in our sample. Because the CPDF does not identify a

specific address or supervisor for incumbents, we forwarded the

questionnaires to the appropriate agency personnel offices for

distribution. Pairs of trained job analysts composed of at least one female

and one minority completed the first set of desk audits for 100 of the 257

positions for which the incumbent and the supervisor returned a

questionnaire.7

 Of the 100 positions, we included in our analyses the 84

positions for which the incumbent and the supervisor provided complete

responses to all questionnaire items. To ensure the independence of the

validation process, the contractor instructed the job analysts not to review

the incumbent/supervisor responses to the questionnaire for any position

before they completed a desk audit.

On the basis of the pilot test, we determined that (1) incumbents at higher

GS grades tended to undervalue their positions when compared to the desk

audit, while incumbents at lower grades tended to overvalue their

positions and (2) the grades of incumbents included in our study were not

evenly distributed across the matrix rows and columns. As a result, the

validity coefficient between GS grades estimated on the basis of the desk

audits versus incumbent/supervisor questionnaire responses was .74.

Because the combination of the two effects threatened to distort the

results of our planned statistical comparisons, we stratified our sample by

classifying the incumbents of the 58 occupations into one of seven groups,

or strata, on the basis of GS grade: grades 1 to 4, grade 5, grade 6, grades 7

to 8, grades 9 to 10, grade 11, and grades 12 to 15. We then determined the

7

Although the majority of the job analysts had extensive desk audit experience, the contractor

provided the analysts with comprehensive training that included writing job descriptions and

classifying jobs through the use of FES.

Page 30 GAO/GGD-96-20 Federal Job Classification
Appendix I 

Questionnaire Development, Sample

Selection, Response Rate Calculation, and

Validation Methodology

number of incumbent/supervisor pairs needed within each matrix cell

using a complex balancing design to ensure that the row and column totals

would be about equal. This stratified sampling strategy enabled us to

balance the grade distribution of incumbents selected within each row and

column of the matrix, and thus, eliminate any grade level effect in

subsequent analyses.

We randomly selected our sample of full-time permanent nonsupervisory

incumbents from the 58 occupations that met the criteria for each matrix

cell and GS grade stratum and distributed the job content questionnaire to

2,233 incumbents and their supervisors. Because of the random selection

process, we did not select incumbents from 3 of the 58 occupational

series—import specialist, education program, and correspondence clerk.

Table I.7 shows the distribution of the 2,233 incumbents in a matrix

configured according to female and minority representation and strata.

Page 31 GAO/GGD-96-20 Federal Job Classification
Appendix I 

Questionnaire Development, Sample

Selection, Response Rate Calculation, and

Validation Methodology

Table I.7: Distribution of the 2,233

Incumbents by Female and Minority

Representation and Strata

Minority representation Female

representation GS grade strata High Medium Low Total

High GS-1 to 4 45 44 1 90

GS-5 65 66 19 150

GS-6 30 30 30 90

GS-7 to 8 30 30 29 89

GS-9 to 10 30 30 12 72

GS-11 3 44 15 62

GS-12 to 15 1 144 1 146

Subtotal 204 388 107 699

Medium GS-1 to 4 30 16 44 90

GS-5 50 50 50 150

GS-6 30 30 30 90

GS-7 to 8 30 30 30 90

GS-9 to 10 30 30 30 90

GS-11 42 28 20 90

GS-12 to 15 115 19 50 184

Subtotal 327 203 254 784

Low GS-1 to 4 15 30 45 90

GS-5 35 34 81 150

GS-6 30 30 30 90

GS-7 to 8 30 30 30 90

GS-9 to 10 30 30 30 90

GS-11 45 18 27 90

GS-12 to 15 34 19 97 150

Subtotal 219 191 340 750

Total 750 782 701 2,233

Source: GAO analysis of data obtained from OPM.

In addition to the sample of 2,233 incumbent and supervisor pairs, we also

selected a supplemental sample of 303 pairs to enable us to complete a

second set of desk audits expeditiously. We selected the supplemental

sample from those incumbents in the 58 occupations who were located in

one of three geographical areas—Washington, D.C., Dayton/Cincinnati,

OH, or Los Angeles, CA—to represent incumbents working in the eastern,

central, and western United States. Because we did not randomly select

the pairs in the supplemental sample from all incumbents working in the

Page 32 GAO/GGD-96-20 Federal Job Classification
Appendix I 

Questionnaire Development, Sample

Selection, Response Rate Calculation, and

Validation Methodology

58 occupations covered by FES, we did not include their questionnaire

responses in our response rate calculation or analyses.

Response Rate

Calculation

We forwarded the job content questionnaires to the appropriate agency

personnel offices for distribution to each of the 2,233

incumbent/supervisor pairs. For 179 pairs, agency officials notified us that

either the incumbent or the supervisor no longer held the position

indicated by the CPDF or were located outside of the United States. As

planned, we eliminated these pairs from our study. Of the remaining 2,054

pairs, we received questionnaires from 1,633 pairs of respondents, for a

response rate of 80 percent. Table I.8 shows the disposition of each of the

2,233 incumbent/supervisor pairs in our sample.

Table I.8: Disposition of

Incumbent/Supervisor Pairs in the

Sample Incumbent/supervisor pairs eliminated from the study

Incumbent resigned or terminated 63

Incumbent transferred to another agency 48

Incumbent in different occupation than one 

identified by the CPDF 29

Incumbent or supervisor retired 26

Incumbent or supervisor outside the United States 3

Incumbent in position not covered by FES 9

Incumbent deceased 1

Subtotal 179a

Incumbent/supervisor pairs in the study

Respondents 1,633

Nonrespondents 421b

Subtotal 2,054

Total incumbent/supervisor pairs in sample 2,233

aOf the 179 incumbent/supervisor pairs, we eliminated 177 pairs from the study on the basis of

factors relating to the incumbent. Two were eliminated on the basis of factors relating to the

supervisor.

bThe 421 pairs of nonrespondents consisted of (1) 349 pairs for which either the incumbent, the

supervisor, or both did not respond to our questionnaire after three follow-up efforts and (2) 72

pairs for which we were unable to forward a questionnaire to either the incumbent, the supervisor,

or both for reasons which did not eliminate the pairs from our study (e.g., either the incumbent or

the supervisor was on extended leave).

Page 33 GAO/GGD-96-20 Federal Job Classification
Appendix I 

Questionnaire Development, Sample

Selection, Response Rate Calculation, and

Validation Methodology

Occupations Included

in the Study Analyses

For 201 of the 1,633 pairs of respondents, either the incumbent, the

supervisor, or both left some questionnaire items unanswered. Therefore,

we included the remaining 1,432 pairs in our preliminary analyses. On the

basis of the incumbent/supervisor questionnaire responses, we estimated

the GS grade for each of the 1,432 positions and compared this grade with

the position’s actual grade. We considered positions to be appropriately

graded when the actual grade equalled the questionnaire grade; otherwise,

we considered the position to be overgraded if the actual grade was higher

than the questionnaire grade; and undergraded if the actual grade was

lower than the questionnaire grade.

When we designed this study, we planned to compare the overgrading and

undergrading among groups of occupations with different gender and

minority representations or among matrix rows and columns. However,

preliminary analyses showed that more variation in overgrading and

undergrading existed within the same row or column rather than between

different rows or columns. For this reason, we analyzed the 37

occupations for which we had received completed questionnaires from at

least 10 or more incumbent/supervisor pairs. We did not include the

remaining 18 occupations in our analyses because we could not obtain

reliable estimates of overgrading and undergrading for an occupation with

less than 10 pairs of respondents.8

 The 37 occupations included 1,358

incumbent/supervisor pairs and represented about 79 percent of the

incumbents covered by FES. Table I.9 shows the gender and minority

representation of the 37 occupations included in our study.

8

The 18 occupational series for which we received completed questionnaires from less than 10

incumbent/supervisor pairs included tax examining, computer clerk and assistant, security

administration, medical technologist, voucher examining, practical nurse, shipment clerical and

assistance, medical supply aide and technician, medical instrument technician, equipment operator,

architecture, mine safety and health, dietician and nutritionist, veterinary medical science, recreation

specialist, language specialist, equal opportunity assistance, and federal retirement benefits.

Page 34 GAO/GGD-96-20 Federal Job Classification
Appendix I 

Questionnaire Development, Sample

Selection, Response Rate Calculation, and

Validation Methodology

Table I.9: Female and Minority

Representation of the 37 Occupations

Included in the Study

Minority representation Female

representation High Medium Low

High Dental assistant Accounting

technician

Dental hygiene

Medical clerk Budget analysis Occupational

therapist

Personnel clerical

and assistance

Nurse

Paralegal specialist

Secretary

Medium Computer operation Accounting Auditing

Equal employment

opportunity

Computer specialist Cartographic

technician

Equal opportunity

compliance

Contracting Industrial hygiene

Mail and file Diagnostic

radiologic

technologist

Public affairs

Nursing assistant Housing

management

Low Border patrol agent Plant protection and

quarantine

Agricultural

management

Security guard Police Chemistry

Forestry

Mechanical

engineering

Quality assurance

Physical science

technician

Safety and

occupational

health management

Soil conservation

Telecommunications

Source: GAO analysis of data obtained from OPM.

Questionnaire

Validation

We validated our job content questionnaire by using a second set of desk

audits. To simplify selecting positions that represented a broad range of

grades, we combined our seven strata into three groups: high (GS-11 to 15),

medium (GS-6 to 10), and low (GS-1 to 5). We then scheduled interviews

with incumbent/supervisor pairs, which allowed us to complete desk

Page 35 GAO/GGD-96-20 Federal Job Classification
Appendix I 

Questionnaire Development, Sample

Selection, Response Rate Calculation, and

Validation Methodology

audits that were distributed evenly across the three GS grade ranges, the

three selected geographical locations, and the nine matrix cells. Again, to

ensure the independence of the validation process, the contractor

instructed the job analysts not to review the incumbent/supervisor

responses to the questionnaire for any position before they completed a

desk audit.

Although we completed 100 desk audits, we validated the questionnaire on

the basis of the 78 positions that represented the 37 occupations included

in our analyses and for which the incumbent and supervisor provided

complete responses to the questionnaire. For the 78 positions, we

completed 24 desk audits for high, 35 for medium, and 19 for low graded

positions; and 37 desk audits in Washington, D.C.; 21 in Los Angeles, CA;

and 20 in Dayton/Cincinnati, OH. Table I.10 shows the distribution of the

78 positions for which a desk audit was completed by occupation and

matrix cell.

Table I.10: Distribution of Desk Audits Completed According to Female and Minority Representation for the 37 Occupations

Included in the Study

Minority representation Female

representation High Medium Low Total

High Dental assistant 0 Accounting

technician

0 Dental

hygiene

8

Medical clerk 3 Budget

analysis

2 Occupational

therapist

1

Personnel

clerical and

assistance

6 Nurse 0

Paralegal

specialist

1

Secretary 3

Subtotal 9 Subtotal 6 Subtotal 9 24

(continued)

Page 36 GAO/GGD-96-20 Federal Job Classification
Appendix I 

Questionnaire Development, Sample

Selection, Response Rate Calculation, and

Validation Methodology

Minority representation Female

representation High Medium Low Total

Medium Computer

operation

2 Accounting 2 Auditing 4

Equal

employment

opportunity

2 Computer

specialist

3 Cartographic

technician

5

Equal

opportunity

compliance

0 Contracting 3 Industrial

hygiene

2

Mail and file 2 Diagnostic

radiologic

technologist

1 Public

affairs

4

Nursing

assistant

1 Housing

management

1

Subtotal 7 Subtotal 10 Subtotal 15 32

Low Border patrol

agent

4 Plant

protection and

quarantine

0 Agricultural

management

0

Security guard 3 Police 6 Chemistry 3

Forestry 0

Mechanical

engineering

3

Quality

assurance

0

Physical

science

technician

2

Safety and

occupational

health

management

1

Soil

conservation

0

Telecommunications 0

Subtotal 7 Subtotal 6 Subtotal 9 22

Total 23 22 33 78

The resulting validity coefficient between GS grades estimated on the basis

of the 78 desk audits versus incumbent/supervisor questionnaire responses

was .80, thus meeting the goal we set when we designed the study. This is

the only classification study we are aware of in which desk audits were

used to validate the questionnaire results. Table I.11 shows the validity

coefficients between GS grades estimated on the basis of desk audits

Page 37 GAO/GGD-96-20 Federal Job Classification
Appendix I 

Questionnaire Development, Sample

Selection, Response Rate Calculation, and

Validation Methodology

versus the questionnaire responses for incumbent/supervisor pairs,

supervisors, and incumbents.

Table I.11: Validity Coefficients for

Grades Estimated on the Basis of the

78 Desk Audits Versus the

Questionnaire Responses for

Incumbent/Supervisor Pairs,

Supervisors Only, and Incumbents

Only

Source of questionnaire responses Validity coefficient

Incumbent/supervisor pairs .80

Supervisors .75

Incumbents .73

Note: While we validated our questionnaire on the basis of the responses from

incumbent/supervisor pairs, we reported the validity coefficients for supervisors only and

incumbents only for the purpose of full disclosure.

For the 37 occupations included in our analyses, we used loglinear

statistical techniques to analyze the odds of individual occupations being

overgraded or undergraded versus appropriately graded (1) in relation to

GS grades and (2) when the female or minority representation of an

occupation was high, medium, or low. The strength of this particular

statistical approach was that multiple variables could be analyzed

simultaneously, thereby enabling us to examine complex relationships in

the data. Appendix II provides more detailed information on our loglinear

methodology, the loglinear models tested, and the results obtained.

Page 38 GAO/GGD-96-20 Federal Job Classification
Appendix II 

Loglinear Methodology and Analysis Results

This appendix provides additional technical detail on our analytical

approach. It contains a general description of loglinear methodology,

describes the variables analyzed, and presents the loglinear models tested

and the results obtained in each analysis.

Data Analysis

Approach

We used loglinear analyses to examine the relationship between the

likelihood of being overgraded or undergraded and (1) the average GS

grade of the sampled incumbents by occupation and (2) female and

minority representation.1

 We first looked at the relationship between the

likelihood of being overgraded or undergraded and female or minority

representation without taking the effect of the average GS grade into

consideration. For each analysis, we considered the preferred model to be

the simplest one that fit the data and could not be significantly improved

by more complex models. The preferred model included those

components that had statistically significant relationships with effects

after we controlled for the influences of other factors. Hence, the

estimates we obtained were net effects determined after the association of

each variable with all other variables had been taken into account or

statistically eliminated.

On the basis of the preferred model, we estimated the direction and

magnitude of the relationships using odds and odds ratios. The odds

indicated the likelihood that an outcome would occur given a particular

factor or combination of factors, and the odds ratios indicated the size of

the effect of the various factors on that likelihood. The more the odds ratio

diverges from 1.0, the stronger the relationship.

Occupations Included

in Our Study

All of our analyses were based on the 37 occupations shown in table II.1.

The table also indicates by occupation whether the female or minority

representation was high, medium, or low; the average GS grade of the

incumbents in the sample; whether the incumbents were overgraded, aptly

graded, or undergraded; and the odds of being either overgraded or

1

We opted to use loglinear models rather than logistic regression models because of the trichotomous

nature (i.e., three characteristics as opposed to being of a continuous nature) of the dependent

variable. Logistic regression models could have been used had we chosen to look separately at the two

odds that we examined, i.e., the odds of overgrading versus aptly grading and the odds of undergrading

versus aptly grading. This would have allowed us to control simultaneously for the average GS grade,

gender, and minority status had we wanted to determine how the relationship between job content and

GS grades assigned using FES varied on the basis of the gender and minority status of incumbents.

Because our objective was to determine whether the relationship varied on the basis of the proportion

of women and minorities in occupations, we chose the simpler modeling strategy.

Page 39 GAO/GGD-96-20 Federal Job Classification
Appendix II 

Loglinear Methodology and Analysis Results

undergraded versus aptly graded.2 For each occupation or group of

occupations, we derived the odds of being overgraded versus aptly graded

by dividing the number of incumbents that were overgraded by the

number aptly graded. The odds of being undergraded versus aptly graded

were similarly calculated.

2

We defined occupations with high, medium, and low female representation as those in which women

represented 70 percent or more, 31 to 69 percent, and 30 percent or less of incumbents, respectively.

We defined occupations with high, medium, and low minority representation as those in which

minorities represented more than 41 percent, 22 to 41 percent, or less than 22 percent of incumbents,

respectively.

On the basis of the average points derived from the questionnaires completed by the incumbent and

the supervisor, we determined the GS grade for each position. We compared that questionnaire grade

with the incumbent’s actual GS grade and considered the position to be aptly, or appropriately, graded

if the questionnaire grade and the actual grade were the same. Otherwise, we considered positions to

be overgraded if the actual grade was higher than the questionnaire grade and undergraded if the

actual grade was lower than the questionnaire grade.

Page 40 GAO/GGD-96-20 Federal Job Classification
Appendix II 

Loglinear Methodology and Analysis Results

Page 41 GAO/GGD-96-20 Federal Job Classification
Appendix II 

Loglinear Methodology and Analysis Results

Table II.1: Female and Minority

Representation, Average GS Grades,

Observed Sample Frequencies, and

Odds of Being Overgraded and

Undergraded Versus Aptly Graded for

37 Occupations Without Controlling

for GS Grade

Female Minority

Representation

High Medium Low High Medium Low

X X

X X

X X

X X

X X

X X

X X

X X

X X

X X

Subtotal for occupations with high female representation

X X

X X

X X

X X

X X

X X

X X

X X

X X

X X

X X

X X

X X

X X

Subtotal for occupations with medium female representation

X X

X X

X X

Page 42 GAO/GGD-96-20 Federal Job Classification
Appendix II 

Loglinear Methodology and Analysis Results

Observed frequencies

Odds

GS

occupational

series

Average GS

grade in

sample

Over

graded

Aptly

graded

Under

graded Total

Over

versus

aptlya

Under

versus

aptlyb

203 Personnel clerical

and assistance

6.81 33 20 16 69 1.63 0.80

679 Medical clerk 4.45 4 3 15 22 1.29 4.43

681 Dental assistant 5.12 2 6 9 17 0.38 1.46

560 Budget analysis 11.98 84 1 1 86 56.33 1.00

318 Secretary 5.40 20 24 38 82 0.84 1.57

610 Nurse 11.20 16 2 2 20 6.60 1.00

950 Paralegal specialist 11.56 16 0 0 16 33.00 1.00

525 Accounting technician 5.62 6 2 5 13 2.60 2.20

682 Dental hygiene 6.56 2 9 48 59 0.26 5.11

631 Occupational

therapist

10.50 3 1 8 12 2.33 5.67

186 68 142 396 2.72 2.08

360 Equal opportunity

compliance

11.86 46 3 0 49 13.29 0.14

260 Equal employment

opportunity

12.26 41 0 1 42 83.00 3.00

332 Computer operations 7.93 36 3 2 41 10.43 0.71

621 Nursing assistant 4.90 8 8 13 29 1.00 1.59

305 Mail and file 4.20 2 5 3 10 0.45 0.64

647 Diagnostic radiologic

technologist

5.94 5 6 23 34 0.85 3.62

334 Computer specialist 9.67 26 3 4 33 7.57 1.29

1102 Contracting 8.57 17 7 4 28 2.33 0.60

510 Accounting 9.20 10 2 3 15 4.20 1.40

1173 Housing management 5.45 1 5 5 11 0.27 1.00

1371 Cartographic

technician

6.04 45 25 29 99 1.78 1.16

1035 Public affairs 10.38 41 5 7 53 7.55 1.36

511 Auditing 10.74 42 3 1 46 12.14 0.43

690 Industrial hygiene 9.74 16 4 7 27 3.67 1.67

336 79 102 517 4.23 1.29

1896 Border patrol agent 10.75 85 4 3 92 19.00 0.78

85 Security guard 5.15 16 9 21 46 1.74 2.26

83 Police 5.34 4 16 38 58 0.27 2.33

(continued)

Page 43 GAO/GGD-96-20 Federal Job Classification
Appendix II 

Loglinear Methodology and Analysis Results

Female Minority

Representation

High Medium Low High Medium Low

X X

X X

X X

X X

X X

X X

X X

X X

X X

X X

Subtotal for occupations with low female representation

Total

Page 44 GAO/GGD-96-20 Federal Job Classification
Appendix II 

Loglinear Methodology and Analysis Results

Observed frequencies

Odds

GS

occupational

series

Average GS

grade in

sample

Over

graded

Aptly

graded

Under

graded Total

Over

versus

aptlya

Under

versus

aptlyb

436 Plant protection and

quarantine

9.13 14 2 0 16 5.80 0.20

1311 Physical science

technician

5.55 31 30 19 80 1.03 0.64

830 Mechanical

engineering

10.95 34 6 3 43 5.31 0.54

1910 Quality assurance 9.95 17 2 3 22 7.00 1.40

460 Forestry 8.25 6 5 9 20 1.18 1.73

1320 Chemistry 10.88 13 2 2 17 5.40 1.00

457 Soil conservation 8.75 10 3 3 16 3.00 1.00

475 Agricultural

management

9.21 9 3 2 14 2.71 0.71

391 Telecommunications 10.64 8 0 3 11 17.00 7.00

18 Safety and

occupational

health management

11.90 9 1 0 10 6.33 0.33

256 83 106 445 3.07 1.28

8.29 778 230 350 1,358 3.38 1.52

Note: Odds were calculated after adding a small constant (0.5) to all cell frequencies so that cells

with zeros could be included in the calculations.

aThe odds of being overgraded versus aptly graded are equal to the number of overgraded

incumbents divided by the number of aptly graded incumbents.

bThe odds of being undergraded versus aptly graded are equal to the number of undergraded

incumbents divided by the number of aptly graded incumbents.

Source: GAO analysis of questionnaire data.

Relationship Between

Overgrading/

Undergrading and

Female or Minority

Representation

Without Controlling

for GS Grade3

While we derived the study results from models fitted to table II.1, we first

considered the relationship between incumbents being overgraded, aptly

graded, or undergraded and female representation (see table II.2). It

should be noted that the effect of the average GS grade was not statistically

eliminated, or controlled for, in this comparison. For high, medium, and

low female representation, overgraded and undergraded incumbents

outnumbered those appropriately graded. We computed the odds as

described above, and these odds indicated the extent to which overgraded

or undergraded incumbents outnumbered aptly graded incumbents. We

3

All of the results reported in this section are provided for illustrative purposes only.

Page 45 GAO/GGD-96-20 Federal Job Classification
Appendix II 

Loglinear Methodology and Analysis Results

computed ratios, sometimes referred to as odds ratios, by dividing the

odds of overgrading or undergrading in one group of occupations by the

corresponding odds in another group. For example, occupations with high

female representation were about 1.62 times more likely to be

undergraded rather than aptly graded when compared with occupations

having medium female representation (2.09 / 1.29 = 1.62). The odds ratios

shown in the table indicate sizable differences in overgrading and

undergrading between occupations with high female representation and

those with medium representation but only slight differences between

those occupations with medium representation and those with low

representation.

Table II.2: Relationship Between the Observed Sample Frequencies and Female Representation Without Controlling for GS

Grade

Observed frequencies

Female

representation Overgraded

Aptly

graded Undergraded Total

Odds of over

versus aptly

graded Ratios

Odds of under

versus aptly

graded Ratios

High 186 68 142 396 2.74 0.64 2.09 1.62

Medium 336 79 102 517 4.25 1.38 1.29 1.01

Low 256 83 106 445 3.08 1.28

Total 778 230 350 1,358

Note: The blank spaces in columns indicate that we did not compute odds or odds ratios.

Table II.2 indicates that all groups of occupations tended to be overgraded

versus aptly or undergraded. The focus of our study was determining

whether occupations with high female representation differed significantly

from those with medium or low female representation. To determine

whether the differences in the sample data shown in table II.2 reflected

“real” differences in the population and not simply chance or sampling

fluctuations, we fit the models shown in the top section of table II.3 to the

data shown in table II.1.

On the basis of the likelihood ratio chi-square values and degrees of

freedom associated with the different models we fit to the observed data,

the preferred model was the third one in the upper and lower sections of

the table.4

 The third model in the upper section of the table indicated a

pronounced tendency for incumbents of jobs with high female

representation to be undergraded. The third model in the lower section of

4

The likelihood ratio chi-square indicates the relative fit of the various models to the data in the tables.

Page 46 GAO/GGD-96-20 Federal Job Classification
Appendix II 

Loglinear Methodology and Analysis Results

the table indicated a pronounced tendency for the incumbents of

occupations with high minority representation to be overgraded.

Table II.3: Hierarchical Models Tested to Examine Relationships Between Observed Sample Frequencies and Female and

Minority Representation

Models tested

Representation

Model

number

Degrees of

freedom

Likelihood ratio

chi-square Probability Pseudo-R2 a

Female F1 {F} {D} 4 37.18 < 0.001 0.000

F2 {F} {D} {FHD} 2 5.63 0.06 0.849

F3 {F} {D} {FHDU} 3 8.57 0.04 0.770

F4 {F} {D} {FD} 0 0.00 1.00 1.000

Minority M1 {M} {D} 4 18.23 < 0.001 0.000

M2 {M} {D} {MHD} 2 0.98 0.61 0.946

M3 {M} {D} {MHDO} 3 1.57 0.67 0.914

M4 {M} {D} {MD} 0 0.00 1.00 1.000

Legend

F = Female representation (high, medium, or low).

D = Difference between actual GS grade and questionnaire grade (overgraded, aptly, or

 undergraded).

M = Minority representation (high, medium, or low).

Note: The subscripts H, U, and O represent dummy variables, which contrast (1) high female or

minority representation with medium and low, (2) undergraded with aptly and overgraded, and

(3) overgraded with aptly and undergraded, respectively.

aPseudo-R2 is calculated, following Goodman (1978), by subtracting the L2 for a given model from

the L2 for the baseline model of independence (e.g., (37.18 - 5.63) / 37.18 = 0.849).

The result indicates the proportion of the variation in the dependent variable (the change in GS

grade or the odds of overgrading or undergrading) that is accounted for by the factor or set of

factors included in the model.

Table II.4 shows the results of fitting the models in the upper section of

table II.3 to the data in table II.2. The odds ratios we derived from the

preferred model (F3) indicated that in the population from which we drew

our sample, incumbents of occupations with high female representation

were twice as likely (i.e., 2.02 times as likely as shown in the table) to be

undergraded compared with either aptly or overgraded, as incumbents of

occupations with low or medium female representation. Again, we

emphasize that the effect of the average GS grade was not statistically

eliminated, or controlled for, when making this comparison.

Page 47 GAO/GGD-96-20 Federal Job Classification
Appendix II 

Loglinear Methodology and Analysis Results

Table II.4: Relationship Between the Expected Frequencies and Female Representation Without Controlling for GS Grade

Expected frequencies

Model

Female

representation Overgraded

Aptly

graded Undergraded Total

Odds of over

versus aptly

graded Ratios

Odds of under

versus aptly

graded Ratios

F1 High 226.87 67.07 102.06 396.00 3.38 1.00 1.52 1.00

Medium 296.19 87.56 133.25 517.00 3.38 1.00 1.52 1.00

Low 254.94 75.37 114.69 445.00 3.38 1.52

Total 778.00 230.00 350.00 1,358.00

F2 High 186.00 68.00 142.00 396.00 2.74 0.75 2.09 1.63

Medium 318.15 87.06 111.78 517.00 3.65 1.00 1.28 1.00

Low 273.85 74.94 96.22 445.00 3.65 1.28

Total 778.00 230.00 350.00 1,358.00

F3 High 196.04 57.96 142.00 396.00 3.38 1.00 2.45 2.02

Medium 312.76 92.46 111.78 517.00 3.38 1.00 1.21 1.00

Low 269.20 79.58 96.22 445.00 3.38 1.21

Total 778.00 230.00 350.00 1,358.00

Note 1: The blank spaces in columns indicate that we did not compute odds or odds ratios.

Note 2: Due to rounding, expected frequencies do not always add to the total.

We also considered the relationship between incumbents being

overgraded, aptly graded, or undergraded and minority representation (see

table II.5). It should be noted that the effect of the average GS grade was

not statistically eliminated, or controlled for, in this comparison. For high,

medium, and low minority representation, the number of overgraded and

undergraded incumbents outnumbered those appropriately graded. The

odds shown in the table indicate the extent to which overgraded or

undergraded incumbents outnumbered aptly graded incumbents.

Occupations with high minority representation were 1.43 times more likely

to be overgraded rather than aptly graded when compared with

occupations having medium minority representation (4.48 / 3.13 = 1.43).

The odds ratios shown in table II.5 indicate pronounced differences in

overgrading and undergrading between occupations with high minority

representation and those with medium representation but only slight

differences between those occupations with medium representation and

those with low representation.

Page 48 GAO/GGD-96-20 Federal Job Classification
Appendix II 

Loglinear Methodology and Analysis Results

Table II.5: Relationship Between the Observed Sample Frequencies and Minority Representation Without Controlling for

GS Grade

Observed frequencies

Minority

representation Overgraded

Aptly

graded Undergraded Total

Odds of over

versus aptly

graded Ratios

Odds of under

versus aptly

graded Ratios

High 273 61 83 417 4.48 1.43 1.36 0.77

Medium 219 70 123 412 3.13 1.08 1.76 1.21

Low 286 99 144 529 2.89 1.45

Total 778 230 350 1,358

Note: The blank spaces in columns indicate where we did not compute odds or odds ratios.

Table II.5 indicates that all groups of occupations tend to be overgraded

versus aptly or undergraded. The focus of our study was determining

whether occupations with high minority representation differed

significantly from those with medium or low minority representation.

Table II.6 shows the results of fitting the models in the lower section of

table II.3 to the data in table II.5. The odds ratios we derived from the

expected frequencies indicate the extent of the differences that existed in

the population. As previously noted, the preferred model selected was

model M3, which indicated that in the population, incumbents of

occupations with high minority representation were 1.64 times more likely

to be overgraded rather than either aptly or undergraded. The effect of the

average GS grade was not statistically eliminated, or controlled for, in this

comparison.

Page 49 GAO/GGD-96-20 Federal Job Classification
Appendix II 

Loglinear Methodology and Analysis Results

Table II.6: Relationship Between the Expected Frequencies and Minority Representation Without Controlling for GS Grade

Expected frequencies

Model

Minority

representation Overgraded

Aptly

graded Undergraded Total

Odds of over

versus aptly

graded Ratios

Odds of under

versus aptly

graded Ratios

M1 High 238.90 70.63 107.47 417.00 3.38 1.00 1.52 1.00

Medium 236.04 69.78 106.19 412.00 3.38 1.00 1.52 1.00

Low 303.06 89.59 136.34 529.00 3.38 1.52

Total 778.00 230.00 350.00 1,358.00

M2 High 273.00 61.00 83.00 417.00 4.48 1.50 1.36 0.86

Medium 221.11 73.99 116.90 412.00 2.99 1.00 1.58 1.00

Low 283.89 95.01 150.10 529.00 2.99 1.58

Total 778.00 230.00 350.00 1,358.00

M3 High 273.00 57.10 86.90 417.00 4.78 1.64 1.52 1.00

Medium 221.11 75.70 115.20 412.00 2.92 1.00 1.52 1.00

Low 283.89 97.20 147.91 529.00 2.92 1.52

Total 778.00 230.00 350.00 1,358.00

Note 1: The blank spaces in columns indicate that we did not compute odds or odds ratios.

Note 2: Due to rounding, expected frequencies do not always add to the total.

Relationship Between

Overgrading/

Undergrading Versus

GS Grade and Female

and Minority

Representation

As depicted in figures II.1 and II.2, we determined through preliminary

analyses that as the average GS grade increased (1) the odds of overgrading

increased significantly and (2) the odds of undergrading decreased

slightly. The trend lines through the scatterplots of points, or occupations,

shown in figures II.1 and II.2 were obtained from a loglinear model, which

allowed the average GS grade to be linearly related to the odds of

overgrading and undergrading.5

5

We derived the estimated odds that define the trend lines in the figures from maximum likelihood

estimation procedures rather than the more common generalized least squares procedures. These

estimated odds that we plotted in the two figures take into consideration the number of sampled

incumbents in each occupation.

Page 50 GAO/GGD-96-20 Federal Job Classification
Appendix II 

Loglinear Methodology and Analysis Results

Figure II.1: Odds of Overgrading by

Average GS Grade Across 37

Occupations

Average GS grade

Odds (log scale)

4 5 6 7 8 9 10 11 12 13

0.1

1

10

100

Note: Each plotted point represents one of the 37 occupations in our study.

Page 51 GAO/GGD-96-20 Federal Job Classification
Appendix II 

Loglinear Methodology and Analysis Results

Figure II.2: Odds of Undergrading by

Average GS Grade Across 37

Occupations

Average GS grade

Odds (log scale)

4 5 6 7 8 9 10 11 12 13

0.1

1

10

100

Note: Each plotted point represents one of the 37 occupations in our study.

Table II.7 shows the loglinear models fit to the data in table II.1 to arrive at

the preferred model—model 11. The first model is the baseline and asserts

that the odds of overgrading and undergrading do not vary across the 37

occupations. Model 2, which allows those odds to vary linearly by average

GS grade, improved significantly upon model 1 and indicates that

67 percent of the variation between the actual GS grades and grades

calculated on the basis of the questionnaire was attributable to the average

GS grade.6 The difference between the pseudo-R2 for model 11 and that for

6

We determine whether one model significantly improves upon another by comparing values of

L2—the likelihood ratio chi-square—which indicates the fit of the models to data or how well the

expected frequencies resulting from the models correspond to the observed or sample frequencies.

Since the difference in L2 values between model 1 and 2 is highly significant given the difference in

degrees of freedom between the two (i.e., 698.33 –228.10 = 470.23 with 72 - 70 = 2 degrees of freedom),

we would choose model 2 as the preferred model of the two and conclude that the linear effect of the

GS grade (which is present in model 2 but not in model 1) is a significant effect or one that cannot be

attributed to sampling fluctuations or chance. Comparisons between other models involve similar

calculations and logic.

Page 52 GAO/GGD-96-20 Federal Job Classification
Appendix II 

Loglinear Methodology and Analysis Results

model 2 indicates that female and minority representation accounted for

about 6 percent of the variation (.731 - .673 = .058).7

 Therefore, model 11,

the preferred model indicates that 73 percent of the variation between the

actual GS grades and grades calculated on the basis of the questionnaire

was attributable to a combination of average GS grade and female and

minority representation. The remaining 27 percent of the variation was not

explained by the variables in our study.

7

The effects of the average GS grade and the female and minority representation operate largely

independent of each other. The effect of female and minority representation would account for

approximately 7 percent of the variation if introduced into the model before the GS grade.

Page 53 GAO/GGD-96-20 Federal Job Classification
Appendix II 

Loglinear Methodology and Analysis Results

Table II.7: Loglinear Models Tested to

Examine the Relationship Between the

Frequencies, the GS Grade, and

Female and Minority Representation

Model

number Models tested

Degrees

of

freedom

Likelihood

ratio

chi-squarea Pseudo-R2 b

1 {O} {D} 72 698.33 0.000

2 {O} {GLD} 70 228.10 0.673

3 {O} {GLD} {FHD} 68 214.64 0.693

4 {O} {GLD} {MHD} 68 202.34 0.710

5 {O} {GLD} {FHD} {MHD} 66 185.26 0.735

6 {O} {GLD} {FHD} {MHD} {FMD} 64 183.16 0.738

7 {O} {GLD} {FHD} {MHD} {MMD} 64 182.77 0.738

8 {O} {GLD} {FHD} {MHD} {FHMHD} 64 181.36 0.740

9 {O} {GLD} {FHD} {MHD} {FHGLD} 64 181.29 0.740

10 {O} {GLD} {FHD} {MHD} {MHGLD} 64 180.04 0.742

11 {O} {GLD} {FHDU} {MHDO} 68 188.11 0.731

Legend

O = Occupation (1 of 37 occupations covered by FES that were included in our study).

D = Difference between actual GS grade and questionnaire grade (overgraded, aptly, or

undergraded).

G = Grade (GS-1 to 15).

F = Female representation (high, medium, or low).

M = Minority representation (high, medium, or low).

Subscript L indicates a linear constraint imposed upon the effect of the GS grade on overgrading

or undergrading.

Subscripts H and M represent dummy variables which contrast high female or minority

representation with medium and low representation, and medium female or minority

representation with high and low representation, respectively.

Subscript U represents a dummy variable which contrasts undergraded with aptly and

overgraded.

Subscript O indicates overgraded with aptly and undergraded.

aThe likelihood ratio chi-square (L2) indicates the relative fit of the various models to the data in

the table.

bPseudo-R2 is calculated, following Goodman (1978), by subtracting the L2 for a given model from

the L2 for the baseline model of independence (e.g., (698.33 - 228.10) / 698.33 = 0.673). The

result indicates the proportion of the variation in the dependent variable (the change in GS grade

or the odds of overgrading or undergrading) that is accounted for by the factor or set of factors

included in the model.

Table II.8 shows the expected frequencies under model 11 and the odds of

overgrading and undergrading derived from them. Across the 37

occupations included in our study, of the total 1,358 positions in our

sample, about 780 or 57 percent appeared to be overgraded, 352 or

Page 54 GAO/GGD-96-20 Federal Job Classification
Appendix II 

Loglinear Methodology and Analysis Results

26 percent appeared to be undergraded, and 232 or 17 percent appeared to

be aptly graded.8

We recognize that, on the surface, these summary results could raise

questions about the overall accuracy of the FES classification system. We

believe that our results should be viewed with caution in this respect

because we did not design our study to assess the overall accuracy of the

classification system. Rather, our use of nontraditional methods, i.e., the

use of the primary standard rather than occupation-specific standards on

which to develop a job content questionnaire coupled with the use of the

questionnaire rather than desk audits, and our sample selection

methodology were designed to examine the relative assignment of grades

among groups of occupations. That is, our study was designed to assess

whether there were differences in the likelihood of overgrading or

undergrading among groups of occupations that included higher or lower

proportions of women and minorities.

We validated our design for achieving this objective; we did not validate

our design for an objective of expressing an opinion on the overall

accuracy of the classification system. Had we undertaken such an

assessment, we would have utilized a more extensive strategy to validate

the relationship between the questionnaire we developed and the results

of more traditional classification tools such as desk audits, or indeed a

heavier reliance directly on desk audits, which is how classification

accuracy studies are usually conducted.

We also recognize that the overall extent of apparent overgrading or

undergrading identified may involve some measurement error. However,

we have no reason to believe that such error would be more pronounced

for any particular group of occupations, for example, occupations with

high female representation compared with those with medium or low

representation. Thus, we do not believe that our estimates of the

differences in odds of overgrading or undergrading for the groups of

occupations included in our analysis have been affected by any possible

measurement error. Also, the likelihood of a position being overgraded,

rather than aptly graded increased as the incumbents’ GS grades increased.

However, the incumbents’ grades had virtually no effect on the likelihood

that a position was undergraded versus aptly graded. Accordingly, in

measuring the odds of overgrading among those groups of occupations,

we controlled for (statistically eliminated the effect of) the grade level

8

Due to rounding, the frequency totals for overgraded, undergraded, and aptly graded positions do not

add to 1,358.

Page 55 GAO/GGD-96-20 Federal Job Classification
Appendix II 

Loglinear Methodology and Analysis Results

effect we observed. The remaining variation in the data indicated

statistically significant differences among the groups of occupations, and

we report on those results.

Page 56 GAO/GGD-96-20 Federal Job Classification
Appendix II 

Loglinear Methodology and Analysis Results

Page 57 GAO/GGD-96-20 Federal Job Classification
Appendix II 

Loglinear Methodology and Analysis Results

Table II.8: Female and Minority

Representation, Average GS Grades,

Expected Frequencies, and Odds of

Overgraded and Undergraded Versus

Aptly Graded for 37 Occupations

Female Minority

Representation

High Medium Low High Medium Low

X X

X X

X X

X X

X X

X X

X X

X X

X X

X X

Subtotal for occupations with high female representation

X X

X X

X X

X X

X X

X X

X X

X X

X X

X X

X X

X X

X X

X X

Subtotal for occupations with medium female representation

X X

X X

X X

Page 58 GAO/GGD-96-20 Federal Job Classification
Appendix II 

Loglinear Methodology and Analysis Results

Expected frequencies

Odds

GS occupational series

Average GS

grade in

sample

Over

graded

Aptly

graded

Under

graded Total

Over

versus

aptlya

Under

versus

aptlyb

203 Personnel clerical and

assistance

6.81 33.93 11.04 24.17 69 3.07 2.19

679 Medical clerk 4.45 4.64 4.88 12.63 22 0.95 2.59

681 Dental assistant 5.12 4.75 3.57 8.83 17 1.33 2.47

560 Budget analysis 11.98 75.78 4.13 6.24 86 18.35 1.51

318 Secretary 5.40 13.94 19.94 48.27 82 0.70 2.42

610 Nurse 11.20 16.67 1.34 2.14 20 12.44 1.60

950 Paralegal specialist 11.56 13.78 0.92 1.44 16 14.98 1.57

525 Accounting technician 5.62 2.46 3.16 7.53 13 0.78 2.38

682 Dental hygiene 6.56 16.46 13.22 29.47 59 1.25 2.23

631 Occupational therapist 10.50 9.31 1.06 1.78 12 8.78 1.68

191.72 63.26 142.50 396 3.04 2.27

360 Equal opportunity

compliance

11.86 46.84 1.24 1.07 49 37.77 0.86

260 Equal employment

opportunity

12.26 40.53 0.88 0.74 42 46.06 0.84

332 Computer operations 7.93 29.40 5.49 6.27 41 5.36 1.14

621 Nursing assistant 4.90 9.61 8.08 11.46 29 1.19 1.42

305 Mail and file 4.20 2.56 3.05 4.54 10 0.84 1.49

647 Diagnostic radiologic

technologist

5.94 9.66 10.57 13.92 34 0.91 1.32

334 Computer specialist 9.67 24.66 4.23 4.27 33 5.83 1.01

1102 Contracting 8.57 17.38 5.15 5.62 28 3.37 1.09

510 Accounting 9.20 10.50 2.27 2.37 15 4.63 1.04

1173 Housing management 5.45 2.59 3.62 4.94 11 0.72 1.36

1371 Cartographic

technician

6.04 29.15 30.33 39.67 99 0.96 1.31

1035 Public affairs 10.38 43.00 5.18 4.97 53 8.30 0.96

511 Auditing 10.74 38.62 3.89 3.64 46 9.93 0.94

690 Industrial hygiene 9.74 20.38 3.38 3.39 27 6.03 1.00

324.88 87.36 106.87 517 3.72 1.22

1896 Border patrol agent 10.75 84.62 3.89 3.63 92 21.75 0.93

85 Security guard 5.15 16.62 12.34 17.19 46 1.35 1.39

83 Police 5.34 12.93 19.04 26.18 58 0.68 1.38

(continued)

Page 59 GAO/GGD-96-20 Federal Job Classification
Appendix II 

Loglinear Methodology and Analysis Results

Female Minority

Representation

High Medium Low High Medium Low

X X

X X

X X

X X

X X

X X

X X

X X

X X

X X

Subtotal for occupations with low female representation

Total

Page 60 GAO/GGD-96-20 Federal Job Classification
Appendix II 

Loglinear Methodology and Analysis Results

Expected frequencies

Odds

GS occupational series

Average GS

grade in

sample

Over

graded

Aptly

graded

Under

graded Total

Over

versus

aptlya

Under

versus

aptlyb

436 Plant protection and

quarantine

9.13 11.07 2.48 2.60 16 4.46 1.05

1311 Physical science

technician

5.55 19.43 25.79 34.93 80 0.75 1.35

830 Mechanical

engineering

10.95 36.74 3.34 3.07 43 11.00 0.92

1910 Quality assurance 9.95 17.08 2.55 2.52 22 6.70 0.99

460 Forestry 8.25 11.62 4.03 4.50 20 2.88 1.12

1320 Chemistry 10.88 14.52 1.37 1.26 17 10.60 0.92

457 Soil conservation 8.75 10.34 2.80 3.02 16 3.69 1.08

475 Agricultural

management

9.21 9.83 2.12 2.21 14 4.68 1.04

391 Telecommunications 10.64 9.25 0.98 0.92 11 9.44 0.94

18 Safety and

occupational

health management

11.90 9.18 0.52 0.45 10 17.65 0.87

263.33 81.25 102.48 445 3.24 1.26

8.29 779.93 231.87 351.85 1,358c 3.37 1.52

aThe odds of being overgraded versus aptly graded are equal to the number of overgraded

incumbents divided by the number of aptly graded incumbents.

bThe odds of being undergraded versus aptly graded are equal to the number of undergraded

incumbents divided by the number of aptly graded incumbents.

cDue to rounding, totals for overgraded, aptly graded, and undergraded do not add to 1,358.

Source: GAO analysis of questionnaire data.

Table II.9 provides coefficients that indicate the direction and magnitude

of the different effects included in model 11. The table shows that the

relationship between the average GS grades of occupations and high

minority representation and the odds of incumbents being overgraded was

statistically significant, while only high female representation was related

in a significant way to the odds of incumbents being undergraded.

The odds ratio 1.64 tells us that as the average GS grade increased across

occupations by one grade, the odds of incumbents being overgraded

increased by a factor of 1.64; that is, the likelihood of being overgraded in

an occupation in which the average grade of incumbents was GS-10 was

Page 61 GAO/GGD-96-20 Federal Job Classification
Appendix II 

Loglinear Methodology and Analysis Results

1.64 times as great as the likelihood of being overgraded in an occupation

in which the average grade was GS-9. Independent of that, occupations

with high minority representation were 2.18 times more likely to be

overgraded than occupations with low or medium minority representation.

Finally, occupations with high female representation were 1.77 times more

likely to be undergraded as occupations with low or medium female

representation. The coefficient for the GS grade effect on undergrading was

not significantly different from 1.0 (which indicates no effect), and the

z-value associated with it implies that the effect can reasonably be due to

chance.9

Table II.9: Coefficients and Odds Ratios Describing the Effects of the Average GS Grade and High Female and Minority

Representation on Overgrading and Undergrading

Factor Coefficienta

Standard

error Z-value Odds ratiob

Lower 95 percent

confidence

interval estimate

Upper 95 percent

confidence

interval estimate

Effect on overgrading

GS grade 0.497 0.038 13.24 1.64 1.53 1.77

High minority

representation

0.780 0.153 5.10 2.18 1.62 2.94

Effect on undergrading

GS grade - 0.071 0.043 - 1.65 0.93 0.86 1.01

High female

representation

0.570 0.145 3.94 1.77 1.33 2.35

Note: Coefficients, standard errors, and z-values are taken from the Loglinear Program in SPSSx.

aCoefficients are logged coefficients and indicate how much the natural logarithm of the odds of

overgrading or undergrading differ across GS grades or female and minority representation.

bOdds ratios are derived directly from coefficients by taking antilogarithms. They are more readily

interpretable than coefficients because they indicate how much the odds of overgrading or

undergrading increase when GS grade increases or when occupations with high female or

minority representation are compared with occupations having low or medium female or minority

representation.

9

Z-values are the ratios of coefficients to their standard errors. When z-values exceed + 1.96 or - 1.96,

coefficients are statistically significant at the 95 percent level of confidence.

Page 62 GAO/GGD-96-20 Federal Job Classification
Appendix III 

Comments From the Office of Personnel

Management

Note: GAO comments

supplementing those in the

report text appear at the

end of this appendix.

See pp. 7-9 and 61-62.

Page 63 GAO/GGD-96-20 Federal Job Classification
Appendix III 

Comments From the Office of Personnel

Management

See p. 11.

See pp. 11-12.

See p. 12.

Page 64 GAO/GGD-96-20 Federal Job Classification
Appendix III 

Comments From the Office of Personnel

Management

See comment 1.

See comment 2.

See pp. 8 and 10.

See pp. 10 and 12.

Page 65 GAO/GGD-96-20 Federal Job Classification
Appendix III 

Comments From the Office of Personnel

Management

See p. 12.

Page 66 GAO/GGD-96-20 Federal Job Classification
Appendix III 

Comments From the Office of Personnel

Management

The following are GAO’s supplemental comments on the Office of

Personnel Management’s letter dated September 8, 1995.

GAO Comments 1. OPM said that some findings in our study, such as the overgrading and

undergrading within the same occupation, were left largely unexplained.

As noted in the text, the job content questionnaire was designed and

validated to achieve our review objective relative to comparing groups of

occupations, and we cannot attest to the questionnaire’s validity when

used across GS grades within occupations, for specific occupations, or for

individual positions.

2. OPM said that several uninvestigated variables were mentioned in our

study, any one of which might account for some or all of the differences

between the study results and the agency classification results. As noted in

the text, we reported that 73 percent of the variation we found between

actual GS grades and those we estimated on the basis of the questionnaire

was attributable to the average GS grades and female or minority

representation; the remaining 27 percent was not explained by the

variables included in our study. Thus, we believe the variables in our study

accounted for most of the variation.

Page 67 GAO/GGD-96-20 Federal Job Classification
Appendix IV 

Major Contributors to This Report

General Government

Division, Washington,

D.C.

Larry H. Endy, Assistant Director, Federal Management and

 Workforce Issues

Brenda J. Bridges, Evaluator-in-Charge

Gregory H. Wilmoth, Senior Social Science Analyst

Douglas M. Sloane, Senior Social Science Analyst

Don D. Allison, Senior Evaluator

Jennifer S. Cruise, Senior Evaluator

Page 68 GAO/GGD-96-20 Federal Job Classification
Bibliography

Aaron, H.J., and C.M. Lougy. The Comparable Worth Controversy.

Washington, D.C.: The Brookings Institute, 1986.

Acker, J. “Sex Bias in Job Evaluation: A Comparable Worth Issue,”

Ingredients for Women’s Employment Policy. Albany: State University of

New York Press, 1987.

Acker, J. Doing Comparable Worth: Gender, Class, and Pay Equity.

Philadelphia: Temple University Press, 1989.

Agarwal, N.C. “Pay Discrimination: A Comparative Analysis of Research

Evidence and Public Policy in Canada, the United States, and Britain.”

Columbia Journal of World Business, Vol. 18 (1983), pp. 28-38.

Agarwal, N.C. “Pay Equity in Canada: Current Developments.” Labor Law

Journal, Vol. 41 (1990), pp. 518-525.

Agassi, J.B. Women on the Job: The Attitudes of Women to Their Work.

Lexington, MA: Lexington Books, 1979.

A Guide to Pay Equity in Local Government. State of Minnesota

Department of Employee Relations. St. Paul, MN: 1990.

Aklin, M.T. Office Job Evaluation. Des Plaines, IL: Industrial Management

Society, 1971.

Aldrich, M., and R. Buchele. The Economics of Comparable Worth.

Cambridge, MA: Ballinger Publishing Company, 1986.

Alexander, R.A., and G.V. Barrett. “Equitable Salary Increase Judgments

Based Upon Merit and Non-merit Considerations: A Cross-national

Comparison.” International Review of Applied Psychology, Vol. 31 (1982),

pp. 443-454.

Algina, J. “Comments on Bartko’s ‘On Various Intraclass Correlation

Reliability Coefficients.’” Psychological Bulletin, Vol. 85 (1978), pp.

135-138.

Allison, E.K. “Sex Linked Differentials in the Beauty Industry.” The Journal

of Human Resources, Vol. 11 (1976), pp. 383-390.

Page 69 GAO/GGD-96-20 Federal Job Classification
Bibliography

Almquist, E.M. Minorities, Gender, and Work. Lexington, MA: Lexington

Books, 1979.

Anderson, C.H. and D.B. Corts. Development of a Framework for a

Factor-ranking Benchmark System for Job Evaluation. U.S. Civil Service

Commission, Personnel Research and Development Center, Project

6B132A. Washington, D.C.: 1973.

Andrews, I.R., and E.R. Valenzi. “Overpay Inequity or Self-Image as a

Worker: A Critical Examination of an Experimental Induction Procedure.”

Organizational Behavior and Human Performance, Vol. 5 (1970), pp.

266-276.

A Job Evaluation Study of Selected Job Classes of the State and Counties

of Hawaii. Arthur Young International. Honolulu, HA: 1987.

Arvey, R.D. “Sex Bias in Job Evaluation Procedures.” Personnel

Psychology, Vol. 39 (1986), pp. 315-335.

Arvey, R.D., and M. Begalla. “Analyzing the Homemaker Job Using the

Position Analysis Questionnaire (PAQ).” Journal of Applied Psychology,

Vol. 60 (1975), pp. 513-517.

Arvey, R.D., et al. “Potential Sources of Bias in Job Analytic Processes.”

Academy of Management Journal, Vol. 25 (1982), pp. 618-629.

Arvey, R.D., and J.A. Fossum. “Application of Personnel Assessment

Concepts and Methods in Job Evaluation Procedures.” Personnel

Assessment Monographs, Vol. 1 (1986).

Arvey, R.D., and K.E. Holt. “Cost Impact of Alternative Comparable Worth

Strategies.” Presented at the 94th Annual Convention of the American

Psychological Association, Washington, D.C.: 1986.

Arvey, R.D., S.E. Maxwell, and L.M. Abraham. “Reliability Artifacts in

Comparable Worth Procedures.” Journal of Applied Psychology, Vol. 70

(1985), pp. 695-705.

Arvey, R.D., E.M. Passino, and J.W. Lounsbury. “Job Analysis Results as

Influenced by Sex of Incumbent and Sex of Analyst.” Journal of Applied

Psychology, Vol. 62 (1977), pp. 411-416.

Page 70 GAO/GGD-96-20 Federal Job Classification
Bibliography

Ash, P. “The Reliability of Job Evaluation Factors.” Journal of Applied

Psychology, Vol. 32 (1948), pp. 313-320.

Ash, P. “A Statistical Analysis of the Navy’s Method of Position

Evaluation.” Public Personnel Review, Vol. 11 (1950), pp. 130-138.

Ash, R.A. “Empirical Validity Evidence for a Task-based Job-component

Method of Job Analysis.” Presented at the Annual Conference of the

International Personnel Management Association Assessment Council,

Orlando, FL: 1989.

Ashenfelter, O., and J.D. Mooney. “Graduate Education, Ability, and

Earnings.” Review of Economics and Statistics, Vol. 50 (1968), pp. 78-86.

Asher, M., and J. Popkin. “The Effect of Gender and Race Differentials on

Public-private Wage Comparisons: A Study of Postal Workers.” Industrial

and Labor Relations Review, Vol. 38 (1984), pp. 16-25.

Astin, H.S., and A.E. Bayer. “Sex Discrimination in Academe.” Educational

Records, Vol. 53 (1972), pp. 101-118.

A Study of Comparable Worth Within the State of Maine Classification and

Compensation System. William M. Mercer-Meidinger, Inc. New York: 1986.

Atchison, T., and W. French. “Pay Systems for Scientists and Engineers.”

Industrial Relations, Vol. 7 (1967), pp. 44-56.

Auster, E.R. “Task Characteristics as a Bridge Between Macro- and

Microlevel Research on Salary Inequality Between Men and Women.”

Academy of Management Review, Vol. 14 (1989), pp. 173-193.

Avolio, B.J., K.G. Kroeck, and B.R. Nathan. “Category Accessibility,

Ratings of Prototypicality, and Perceptions of Managerial Role Attributes.”

Psychological Reports, Vol. 62 (1988), pp. 195-210.

Azevedo, R.E., and L. Roth. “Canadian-United States Experience with

Comparable Worth: The View From Minnesota.” Labor Law Journal, Vol.

41 (1990), pp. 531-534.

Baer, J.A. The Chains of Protection. Westport, CT: Greenwood Press, 1978.

Page 71 GAO/GGD-96-20 Federal Job Classification
Bibliography

Bailey, W.R., and A.E. Schwenk. “Wage Rate Variation by Size of

Establishment.” Industrial Relations, Vol. 19 (1980), pp. 192-198.

Barnes, W.E., and F.B. Jones. “Differences in Male and Female Quitting.”

Journal of Human Resources, Vol. 9 (1974), pp. 439-451.

Barnett, E. “Comparable Worth and the Equal Pay Act: Proving Sex-based

Wage Discrimination Claims after County of Washington v. Gunther.”

Wayne Law Review, Vol. 28 (1982), pp. 1669-1700.

Baron, J.M., A. Davis-Blake, and W.T. Bielby. “The Structure of

Opportunity: How Promotion Ladders Vary Within and Among

Organizations.” Administrative Science Quarterly, Vol. 31 (1986), pp.

248-273.

Barrett, G.V. “Salary Increase Models: A Cross-national Comparison.”

Research Seminar on Comparitive Managerial Behavior Research at the

International Institute of Management, Berlin: 1971.

Barrett, G.V. “Job Evaluation as a Fair and Effective Technique for

Establishing Pay.” Presented at the Personnel/Human Resource

Management Division, 1990 National Meeting of the Academy of

Management, San Francisco, CA: 1990.

Barrett, G.V. “Comparison of Skill-based Pay with Traditional Job

Evaluation Techniques.” Human Resource Management Review, Vol. 1

(1991), pp. 97-105.

Barrett, G.V. “Task Design, Individual Attributes, Work Satisfaction, and

Productivity.” Work Organization Research: American and European

Perspectives. Kent, OH: Kent State University Press, 1978.

Barrett, G.V., et al. “Frequently Encountered Problems in the Application

of Regression Analysis to the Investigation of Sex Discrimination in

Salaries.” Public Personnel Management, Vol. 15 (1986), pp. 143-157.

Barrett, G.V., and B.M. Bass. “Cross-cultural Issues in Industrial and

Organizational Psychology,” Handbook of Industrial and Organizational

Psychology. Chicago: Rand McNally College Publishing Co, 1976.

Page 72 GAO/GGD-96-20 Federal Job Classification
Bibliography

Barrett, G.V., F.H. Dambrot, and G.R. Smith. The Relationship Between

Individual Attributes and Job Design: Review and Annotated Bibliography.

University of Akron, Department of Pyschology, Technical Report 6.

Akron, OH: 1975.

Barrett, G.V., and D. Doverspike. “Fundamental Errors and Erroneous

Beliefs in Using Regression Analysis as a Test for Sex Discrimination.”

Law and Psychology Review, Vol. 13 (1989), pp. 1-24.

Barrett, G.V., and D. Doverspike. “Another Defense of Point-factor Job

Evaluation.” Personnel, Vol. 66 (1989), pp. 33-36.

Barrett, G.V., and S.B. Morris. “The American Psychological Association’s

Amicus Curiae Brief in Price Waterhouse v. Hopkins: The Values of

Science Versus the Values of the Law,” Law and Human Behavior, Vol. 17

(1993), pp. 201-215.

Barrett, G.V., and D.M. Sansonetti. “Issues Concerning the Use of

Regression Analysis in Salary Discrimination Cases.” Personnel

Psychology, Vol. 41 (1988), pp. 503-516.

Barrett, G.V., W.S. Whittaker, and R.A. Alexander. “Equitable Salary

Increases: A Cross-national Comparison.” Abstract Guide of 20th

International Congress of Psychology, Tokyo: 1972.

Bar-tal, D., et al., eds. Stereotyping and Prejudice, New York:

Springer-Verlag, 1989.

Basow, S.A. Gender Stereotypes: Traditions and Alternatives, 2nd ed.

Pacific Grove, CA: Brooks/Cole Publishing Co., 1986.

Bass, B.M., and B.J. Avolio. The Transformational and Transactional

Leadership Behavior of Management Men and Women as Described by the

Men and Women Who Directly Report to Them. University of New York at

Binghamton, Center for Leadership Studies/School of Management, Report

91-3. Binghamton, NY: 1991.

Beatty, R.W., and J.R. Beatty. “Job Evaluation and Discrimination: Legal,

Economic, and Measurement Perspectives on Comparable Worth and

Women’s Pay,” Women In The Workforce. New York: Praeger Publishers,

1982.

Page 73 GAO/GGD-96-20 Federal Job Classification
Bibliography

Beatty, R.W., and J.R. Beatty. “Some Problems with Contemporary Job

Evaluation Systems,” Comparable Worth and Wage Discrimination:

Technical Possibilities and Political Realities. Philadelphia: Temple

University Press, 1984.

Becker, M.E. “Comparable Worth in Antidiscrimination Legislation: A

Reply to Freed and Polsby.” University of Chicago Law Review, Vol. 51

(1984), pp. 1112-1134.

Belcher, D.W. “Pay Equity or Pay Fairness?” Compensation Review, Vol. 11

(1979), pp. 31-37.

Bellace, J.R. “The Impact of the American and British Equal Pay Guarantee

on Job Evaluation.” Applied Psychology An International Review, Vol. 36

(1987), pp. 9-24.

Bellak, A.O., M.W. Bates, and D.M. Glasner. “Job Evaluation: Its Role in the

Comparable Worth Debate.” Public Personnel Management, Vol. 12 (1983),

pp. 418-424.

Bellar, A.H. “The Economics of Enforcement of an Antidiscrimination

Law: Title VII of the Civil Rights Act of 1964.” Journal of Law and

Economics, Vol. 21 (1978), pp. 359-380.

Bellows, R.M., and M.F. Estep. “Job Evaluation Simplified: The Utility of

the Occupational Characteristics Checklist.” Journal of Applied

Psychology, Vol. 32 (1948), pp. 354-359.

Benge, E.J. “Statistical Study of a Job Evaluation Point System.” Modern

Management, (1947), pp. 17-23.

Benge, E.J., S.L.H. Burk, and E.N. Hay. Manual of Job Evaluation:

Procedures of Job Analysis and Appraisal. New York and London: Harper

and Brothers, 1941.

Benson, P.G., and J.S. Hornsby. “The Politics of Pay: The Use of Influence

Tactics in Job Evaluation Committees.” Group and Organization Studies,

Vol. 13 (1988), pp. 208-224.

Benson, P.G., and J.S. Hornsby. “Job Evaluation Committees as Small

Groups: Implications of Group Dynamics for Fairness in Pay.”

International Journal of Public Administration, Vol. 14 (1991), pp. 845-869.

Page 74 GAO/GGD-96-20 Federal Job Classification
Bibliography

Berelson, B. Content Analysis in Communication Research. Glencoe, IL:

Folcroft Press, 1970.

Bergmann, B.R. The Economic Emergence of Women. New York: Basic

Books, 1986.

Berheide, C.W., et al. Minorities and Pay Equity in New York State. Center

for Women in Government, Working Paper 17. Albany, NY: 1986.

Berheide, C.W., et al. “A Pay Equity Analysis of Female-dominated and

Disproportionately Minority New York State Job Titles.” Humanity and

Society, Vol. 11 (1987), pp. 465-485.

Bertaux, N.E. “The Roots of Today’s ‘Women’s Jobs’ and ‘Men’s Jobs’:

Using the Index of Dissimilarity Segregation by Gender.” Explorations in

Economic History, Vol. 28 (1991), pp. 433-459.

Betz, N.E., and G. Hackett. “The Relationship of Career-related

Self-efficacy Expectations to Perceived Career Options in College Women

and Men.” Journal of Counseling Psychology, Vol. 28 (1981), pp. 399-410.

Beuhring, T. “Designing a New Job Evaluation System Based on Employee

Input.” Presented at the 94th Annual American Psychological Association

Conference, Washington, D.C.: 1986.

Beuhring, T. “Incorporating Employee Values in Job Evaluation.” Journal

of Social Issues, Vol. 45 (1989), pp. 169-189.

Beutell, N.J., and O.C. Brenner. “Sex Differences in Work Values.” Journal

of Vocational Behavior, Vol. 28 (1986), pp. 29-41.

Bielby, W.T., and J.N. Baron. “Undoing Discrimination: Job Integration and

Comparable Worth,” Ingredients for Women’s Employment Policy. Albany:

State University of New York Press, 1987.

Bigoness, W.J. “Sex Differences in Job Attribute Preferences.” Journal of

Organizational Behavior, Vol. 9 (1988), pp. 139-147.

Birnbaum, M.H. “Procedures for the Detection and Correction of Salary

Inequities,” Salary Equity: Detecting Sex Bias in Salaries Among College

and University Professors. Lexington, MA: Lexington Books, 1979.

Page 75 GAO/GGD-96-20 Federal Job Classification
Bibliography

Birnbaum, M.H. “Perceived Equity of Salary Policies.” Journal of Applied

Psychology, Vol. 68 (1983), pp. 49-59.

Birnbaum, M.H. “Relationships Among Models of Salary Bias.” American

Psychologist, Vol. 40 (1985), pp. 862-866.

Blau, F.D. Equal Pay in the Office. Lexington, MA: Lexington Books, 1977.

Blau, F.D., and L.M. Kahn. “Race and Sex Differences in Quits by Young

Workers.” Industrial and Labor Relations Review, Vol. 34 (1981), pp.

563-577.

Blaxall, M., and B. Reagan, eds. Women and the Workplace. Chicago: The

University of Chicago Press, 1976.

Blinder, A.S., ed. Paying for Productivity. Washington, D.C.: The Brookings

Institute, 1990.

Blood, M.R., W.K. Graham, and S. Zedeck. “Resolving Compensation

Disputes With Three-party Job Evaluation.” Applied Psychology: An

International Review, Vol. 36 (1987), pp. 39-50.

Blum, L.M. “Possibilities and Limits of the Comparable Worth Movement.”

Gender and Society, Vol. 1 (1987), pp. 380-399.

Blum, L.M. Between Feminism and Labor: The Significance of the

Comparable Worth Movement. Berkeley, CA: University of California

Press, 1991.

Blumrosen, A.W. “The Group Interest Concept, Employment

Discrimination, and Legislative Intent: The Fallacy of Connecticut v. Teal.”

Harvard Journal on Legislation, Vol. 20 (1983), pp. 99-135.

Blumrosen, R.G. “Wage Discrimination, Job Segregation, and Title VII of

the Civil Rights Act of 1964.” Journal of Law Reform, Vol. 12 (1979), pp.

399-502.

Blumrosen, R.G. “Wage Discrimination and Job Segregation: The Survival

of a Theory.” University of Michigan Journal of Law Reform, Vol. 14

(1980), pp. 1-14.

Page 76 GAO/GGD-96-20 Federal Job Classification
Bibliography

Blumrosen, R.G. “Wage Discrimination, Job Segregation, and Women

Workers.” Employee Relations Law Journal, Vol. 6 (1980), pp. 77-136.

BNA. “Changing Pay Practices: New Developments in Employee

Compensation.” Labor Relations Week, Vol. 2 (1988), pp. 1-220.

Boardman, S.K., C.C. Harrington, and S.V. Horowitz. “Successful Women:

A Psychological Investigation of Family Class and Education Origins,”

Women’s Career Development. Newbury Park, CA: Sage, 1987.

Booker, S., and L.C. Nuckolls. “Legal and Economic Aspects of

Comparable Worth.” Public Personnel Management, Vol. 15 (1986), pp.

189-206.

Borjas, G.J. “Job Mobility and Earnings Over the Life Cycle.” Industrial and

Labor Relations Review, Vol. 34 (1981), pp. 365-376.

Borjas, G.J. “The Measurement of Race and Gender Wage Differentials:

Evidence From the Federal Sector.” Industrial and Labor Relations

Review, Vol. 37 (1983), pp. 79-91.

Bose, C., and G. Spitze, eds. Ingredients for Women’s Employment Policy.

Albany, NY: State University of New York Press, 1987.

Bowey, A.M., R. Thorpe, and P. Hellier. Payment Systems and Productivity.

New York: St. Martin’s Press, 1986.

Brennan, E.J. “Comparable Worth: Employers Can No Longer Pass the

Buck.” Personnel Journal, Vol. 64 (1985), pp. 110-111.

Brenner, O.C. “Relationship of Education to Sex, Managerial Status, and

the Managerial Stereotype.” Journal of Applied Psychology, Vol. 67 (1982),

pp. 380-383.

Brittingham, B.E., et al. “A Multiple Regression Model for Predicting Men’s

and Women’s Salaries in Higher Education,” Salary Equity: Detecting Sex

Bias in Salaries Among College and University Professors. Lexington, MA:

Lexington Books, 1979.

Bronstein, P., et al., eds. “Stepping on the Academic Career Ladder: How

Are Women Doing?” Women’s Career Development. Newbury Park, CA:

Sage, 1987.

Page 77 GAO/GGD-96-20 Federal Job Classification
Bibliography

Brooks, L. “Sexist Language in Occupational Information: Does It Make a

Difference?” Journal of Vocational Behavior, Vol. 23 (1983), pp. 227-232.

Brown, H.P. The Inequality of Pay. Berkeley, CA: University of California

Press, 1977.

Brown, K.A., and D.L. Huber. “Lowering Floors and Raising Ceilings: A

Longitudinal Assessment of the Effects of an Earnings-at-risk Plan on Pay

Satisfaction.” Personnel Psychology, Vol. 45 (1992), pp. 279-311.

Browne, K.R. “Comparable Worth: An Impermissible Form of Affirmative

Action?” Loyola of Los Angeles Law Review, Vol. 22 (1989), pp. 717-759.

Browne, M.N., and B. Powers. “Henry George and Comparable Worth:

Hypothetical Markets as a Stimulus for Reforming the Labor Market.”

American Journal of Economics and Sociology, Vol. 47 (1988), pp. 461-471.

Buchele, R., and M. Aldrich. “How Much Difference Would Comparable

Worth Make?” Industrial Relations, Vol. 24 (1985), pp. 222-233.

Buckley, J.E. “Pay Differences Between Men and Women in the Same Job.”

Monthly Labor Review, Vol. 94 (1971), pp. 36-39.

Buel, W.D. “A Simplification of Hay’s Method of Recording Paired

Comparisons.” Journal of Applied Psychology, Vol. 44 (1960), pp. 347-348.

Buford, J.A., and D.R. Norris. “A Salary Equalization Model: Identifying

and Correcting Sex-based Salary Differences.” Employee Relations Law

Journal, Vol. 6 (1981), pp. 406-421.

Bungel, J.H. “To Each According to Her Worth?” The Public Interest, Vol.

67 (1982), pp. 77-93.

Callahan-Levy, C.M. “Sex Differences in the Allocation of Pay.” Journal of

Personality and Social Psychology, Vol. 37 (1979), pp. 43-446.

Camden, C.T., and C.W. Kennedy. “Manager Communicative Style and

Nurse Morale.” Human Communication Research, Vol. 12 (1986), pp.

551-563.

Page 78 GAO/GGD-96-20 Federal Job Classification
Bibliography

Campion, M.A., and C.J. Berger. “Conceptual Integration and Empirical

Test of Job Design and Compensation Relationships.” Personnel

Psychology, Vol. 43 (1990), pp. 525-553.

Campion, M.A., R.G. Lord, and E.D. Pursell. “Individual and Organizational

Correlates of Promotion Refusal.” Journal of Vocational Behavior, Vol. 19

(1981), pp. 42-49.

Cannings, K. “An Interdisciplinary Approach to Analyzing the Managerial

Gender Gap.” Human Relations, Vol. 44 (1991), pp. 679-695.

Cappelli, P., and W.F. Cascio. “Why Some Jobs Command Wage Premiums:

A Test of Career Tournament and Internal Labor Market Hypotheses.”

Academy of Management Journal, Vol. 34 (1991), pp. 848-868.

Carey, J.F. “Participative Job Evaluation.” Compensation Review, Vol. 9

(1977), pp. 29-38.

Carlisi, A.M. “The Influence of Sex Stereotyping and the Sex of the Job

Evaluator on Job Evaluation Ratings.” Ph.D. Dissertation, University of

Akron, OH: 1984.

Carney, T.F. Content Analysis: A Technique for Systematic Inference From

Communications. Winnipeg: University of Manitoba Press, 1972.

Carter, R.D., et al. “Multivariate Alternatives to Regression Analysis in the

Evaluation of Salary Equity-parity.” Research in Higher Education, Vol. 20

(1984), pp. 167-179.

Cascio, W.F. Applied Psychology in Personnel Management, 3rd ed.

Englewood Cliffs, NJ: Prentice-Hall, Inc., 1987.

Cellar, D.F., et al. “The Effects of Rater Training, Job Analysis Format and

Congruence of Training on Job Evaluation Ratings.” Journal of Business

and Psychology, Vol. 3 (1989), pp. 387-401.

Cellar, D.F., et al. “The Effect of Field Independence, Job Analysis Format,

and Sex of Rater on the Accuracy of Job Evaluation Ratings.” Journal of

Applied Social Psychology, Vol. 19 (1989), pp. 363-376.

Page 79 GAO/GGD-96-20 Federal Job Classification
Bibliography

Cellar, D.F., M.C. Kernan, and G.V. Barrett. “Conventional Wisdom and

Ratings of Job Characteristics: Can Observers be Objective?” Journal of

Management, Vol. 11 (1985), pp. 131-138.

Centers, R., and D.E. Bugental. “Intrinsic and Extrinsic Job Motivations

Among Different Segments of the Working Population.” Journal of Applied

Psychology, Vol. 50 (1966), pp. 193-197.

Centra, J.A. Women, Men, and a Doctorate. Princeton, NJ: Educational

Testing Service, 1974.

Chacko, T.I. “Women and Equal Employment Opportunity: Some

Unintended Effects.” Journal of Applied Psychology, Vol. 67 (1982), pp.

119-123.

Charles, A.W. “Installing Single-factor Job Evaluation.” Compensation

Review, Vol. 3 (1971), pp. 9-21.

Chesler, D.J. “Reliability and Comparability of Different Job Evaluation

Systems.” Journal of Applied Psychology, Vol. 32 (1948), pp. 465-475.

Chesler, D.J. “Reliability of Abbreviated Job Evaluation Scales.” Journal of

Applied Psychology, Vol. 32 (1948), pp. 622-628.

Chesler, D.J. “Abbreviated Job Evaluation Scales Developed on the Basis

of ‘Internal’ and ‘External’ Criteria.” Journal of Applied Psychology, Vol. 33

(1949), pp. 151-157.

Chewning, M.F., and W.E. Walker. “Sex-typing of Tasks and Occupations.”

Psychological Reports, Vol. 47 (1980), pp. 696-698.

Chi, K.S. “Comparable Worth: Implications of the Washington Case.” State

Government, Vol. 57 (1984), pp. 34-45.

Chiswick, B.R. “Immigrant Earnings Patterns by Sex, Race and Ethnic

Groupings.” Monthly Labor Review, Vol. 103 (1980), pp. 22-25.

Clauss, C.A. “Comparable Worth—The Theory, Its Legal Foundation, and

the Feasibility of Implementation.” University of Michigan Journal of Law

Reform, Vol. 20 (1986), pp. 7-97.

Page 80 GAO/GGD-96-20 Federal Job Classification
Bibliography

Cleveland, J.N., and G. Hollman. “The Effects of the Age-type of Tasks and

Incumbent Age Composition on Job Perceptions.” Journal of Vocational

Behavior, Vol. 36 (1990), pp. 181-194.

Coch, J.V. “Student Choice of Undergraduate Major Field of Study and

Private Internal Rates of Return.” Industrial and Labor Relations Review,

Vol. 26 (1972), pp. 680-685.

Cohen, L. “More Reliable Job Evaluation.” Personnel Psychology, Vol. 4

(1948), pp. 457-464.

Cohen, M.S. “Sex Differences in Compensation.” The Journal of Human

Resources, Vol. 6 (1971), pp. 434-447.

Cole, J.R. Fair Science: Women in the Scientific Community. New York:

Free Press, 1979.

Collett, M.J. “Comparable Worth: An Overview.” Public Personnel

Management, Vol. 12 (1983), pp. 325-331.

Collins, J.M., and P.M. Muchinsky. “An Assessment of the Construct

Validity of Three Job Evaluation Methods: A Field Experiment.” Academy

of Management Journal, Vol. 36 (1993), pp. 895-904.

Collins, R. “Functional and Conflict Theories of Educational

Stratification.” American Sociological Review, Vol 36 (1971), pp.

1002-1019.

Comparable Worth: An Analysis and Recommendations. U.S. Commission

on Civil Rights. Washington, D.C.: 1985.

Comparable Worth: A Symposium on the Issues and Alternatives. Equal

Employment Advisory Council. Washington, D.C.: 1981.

Comparable Worth for Federal Jobs: A Wrong Turn Off the Road Toward

Pay Equity and Women’s Career Advancement. U.S. Office of Personnel

Management. Washington D.C.: 1987.

Comparable Worth in Montana State Government: A Report to the 52nd

Legislature. Montana State, State Personnel Division, Department of

Administration. Helena, MT: 1991.

Page 81 GAO/GGD-96-20 Federal Job Classification
Bibliography

Comparable Worth: Issue for the 80s: A Consultation of the U.S.

Commission on Civil Rights. U.S. Commission on Civil Rights. Washington,

D.C.: 1984.

Converse, J.M., and S. Presser. Survey Questions: Handcrafting the

Standardized Questionnaire. Beverly Hills, CA: Sage Publications, 1986.

Cooper, C.L., and M.J. Davidson. “The High Cost of Stress on Women

Managers.” Organizational Dynamics, Vol. 10 (1982), pp. 44-53.

Cooper, E.A., and G.V. Barrett. “Equal Pay and Gender: Implications of

Court Cases for Personnel Practices.” Academy of Management Review,

Vol. 9 (1984), pp. 84-94.

Cooper, E.A., D. Doverspike, and G.V. Barrett. “Comparison of Different

Methods of Determining the Sex Type of an Occupation.” Psychological

Reports, Vol. 57 (1985), pp. 747-750.

Cooper, E.A., et al. “Sex Bias in Job Evaluation: The Effect of Sex on

Judgments of Factor and Level Weights.” Educational and Psychological

Measurement, Vol. 47 (1987), pp. 369-375.

Cooper, E.A., and R.W. Scholl. “Reliability of Job Evaluation: Differences

Across Sex-typed Jobs.” Journal of Business and Psychology, Vol. 4 (1989),

pp. 155-165.

Cooper, H.M., J.M. Burger, and T.L. Good. “Gender Differences in the

Academic Locus of Control Beliefs of Young Children.” Journal of

Personality and Social Psychology, Vol. 40 (1981), pp. 562-572.

Cortis, L.E. “Psychological Factors in Job Evaluation.” South African

Journal of Psychology, Vol. 2 (1972), pp. 55-66.

Craver, G. “Survey of Job Evaluation Practices in State and County

Governments.” Public Personnel Management, Vol. 6 (1977), pp. 121-131.

Cummings, L.L. “Compensation, Culture, and Motivation.” Organizational

Dynamics, Vol. 12 (1984), pp. 33-44.

Curry, E.W., and D. Walling. “Occupational Prestige: Exploration of a

Theoretical Basis.” Journal of Vocational Behavior, Vol. 25 (1984), pp.

124-138.

Page 82 GAO/GGD-96-20 Federal Job Classification
Bibliography

Dackawich, J., J. Best, and W. York. “Occupational and Educational

Expectations: Sex and Ethnic Variations.” Paper, California State

University, 1978.

Dadoy, M. “Role et Place de l’Analyse du Travail Dans les Systemes

d’Evaluation de la Qualification du Travail.” Le Travail Humain, Vol. 54

(1991), pp. 97-112.

Daniels, H.W. “Winning Acceptance for the Job Evaluation Plan.”

Personnel, Vol. 30 (1953), pp. 30-32.

Danielson, J.L., and R. Smith. “The Application of Regression Analysis to

Equality and Merit in Personnel Decisions.” Public Personnel Management

Journal, Vol. 10 (1981), pp. 126-131.

Darlington, R.B. “Multiple Regression in Psychological Research and

Practice.” Psychological Bulletin, Vol. 69 (1968), pp. 161-182.

Davies, M.W. Women’s Place Is at the Typewriter: Office Work and Office

Workers 1870-1930. Philadelphia: Temple University Press, 1982.

Davis, J.C., and C.M. Hubbard. “On the Measurement of Discrimination

Against Women.” American Journal of Economics and Sociology, Vol. 38

(1979), pp. 287-292.

Davis, K.R. and W.I. Sauser. “Effects of Alternative Weighting Methods in a

Policy-capturing Approach to Job Evaluation: A Review and Empirical

Investigation.” Personnel Psychology, Vol. 44 (1991), pp. 85-127.

Davis, K.R., and W.I. Sauser. “A Comparison of Factor Weighting Methods

in Job Evaluation: Implications for Compensation Systems.” Public

Personnel Management, Vol. 22 (1993), pp. 91-106.

Davis, M.K., and J. Tiffin. “Cross Validation of an Abbreviated Point Job

Evaluation System.” Journal of Applied Psychology, Vol. 34 (1950), pp.

225-228.

Dean, J.W., Jr., and D.J. Brass. “Social Interaction and the Perception of

Job Characteristics in an Organization.” Human Relations, Vol. 38 (1985),

pp. 571-582.

Page 83 GAO/GGD-96-20 Federal Job Classification
Bibliography

De Corte, W. “Estimating Sex-related Bias in Job Evaluation.” Journal of

Occupational and Organizational Psychology, Vol. 66 (1993), pp. 83-96.

DeGooyer, J., and L. Westley. Women’s Work: Undervalued, Underpaid.

National Commission on Working Women. Washington, D.C.: 1984.

Delaney, H.D., R.D. Norman, and D.A. Miller. “An Exploration of the Verbal

Encodability Hypothesis for Sex Differences in the Digit-symbol

(Symbol-digit) Test.” Intelligence, Vol. 5 (1981), pp. 199-208.

DeNisi, A.S., E.T. Cornelius, III., and A.G. Blencoe. “Further Investigation

of Common Knowledge Effects on Job Analysis Ratings.” Journal of

Applied Psychology, Vol. 72 (1987), pp. 262-268.

Deremer, C., D.R. Quaries, and C.M. Temple. “The Success Rate of

Personal Salary Negotiations: A Further Investigation of Academic Pay

Differentials by Sex.” Research in Higher Education, Vol. 16 (1986), pp.

139-154.

Dertien, M.G. “The Accuracy of Job Evaluation Plans.” Personnel Journal,

Vol. 60 (1981), pp. 566-570.

Desmond, R.E., and D.J. Weiss. “Worker Estimation of Ability

Requirements of Their Jobs.” Journal of Vocational Behavior, Vol. 7 (1975),

pp. 23-27.

De Sola Pool, I., ed. Trends in Content Analysis. Urbana, IL: University of

Illinois Press, 1959.

DeVader, C.L. “A Comparison of Three Category Types and Their

Applicability to I/O Rating.” Ph.D. Dissertation, University of Akron, 1986.

Diamond, E.E. “Theories of Career Development and the Reality of

Women at Work,” Women’s Career Development. Newbury Park, CA: Sage,

1987.

Dick, A.H. “Job Evaluation’s Role in Employee Relations.” Personnel

Journal, Vol. 53 (1974), pp. 176-179.

Dictionary of Occupational Titles, 4th ed. U.S. Department of Labor.

Washington, D.C.: 1991.

Page 84 GAO/GGD-96-20 Federal Job Classification
Bibliography

Dillingham, A.E. “Sex Differences in Labor Market Injury Risk.” Industrial

Relations, Vol. 20 (1981), pp. 117-122.

Dolton, P.J. “Testing for Sex Discrimination in Maximum Likelihood

Models.” Applied Economics, Vol. 16 (1984), pp. 225-235.

Dornstein, M. “Perceptions Regarding Standards for Evaluating Pay Equity

and Their Determinants.” Journal of Occupational Psychology, Vol. 58

(1985), pp. 321-330.

Dornstein, M. “Pay Equity Evaluations of Occupations and Their Bases.”

Journal of Applied Social Psychology, Vol. 18 (1988), pp. 905-924.

Dornstein, M. “The Fairness Judgments of Received Pay and Their

Determinants.” Journal of Occupational Psychology, Vol. 62 (1989), pp.

287-299.

Doverspike, D. “A Statistical Analysis of Internal Sex Bias in a Job

Evaluation Instrument (Doctoral dissertation, The University of Akron,

1983).” Dissertation Abstracts International, Vol. 43 (1983), p. 3063B.

Doverspike, D., and G.V. Barrett. “An Internal Bias Analysis of a Job

Evaluation Instrument.” Journal of Applied Psychology, Vol. 69 (1984), pp.

648-662.

Doverspike, D., et al. “Generalizability Analysis of a Point-method Job

Evaluation Instrument.” Journal of Applied Psychology, Vol. 68 (1983), pp.

476-483.

Doverspike, D., et al. “Sex Differences in Short-term Memory Processing.”

Perceptual and Motor Skills, Vol. 58 (1984), pp. 135-139.

Drazin, R. and E.R. Auster. “Wage Differences Between Men and Women:

Performance Appraisal Ratings vs. Salary Allocation as the Locus of Bias.”

Human Resource Management, Vol. 26 (1987), pp. 157-168.

Driver, R.S. “A Case History in Merit Rating.” Personnel, Vol. 13 (1939), pp.

262-269.

Driver, R.S. “The Validity and Reliability of Ratings.” Personnel, Vol. 16

(1940), pp. 137-191.

Page 85 GAO/GGD-96-20 Federal Job Classification
Bibliography

Dunham, R.B. “Relationships of Perceived Job Design Characteristics to

Job Ability Requirements and Job Value.” Journal of Applied Psychology,

Vol. 62 (1977), pp. 760-763.

Dunson, B.H. “Pay, Experience, and Productivity: The Government-Sector

Case.” The Journal of Human Resources, Vol. 20 (1985), pp. 153-160.

Durio, H.F., and C.A. Kidlow. “The Nonretention of Capable Women

Engineering Students.” Research in Higher Education, Vol. 13 (1980), pp.

61-71.

Dyer, L., D.P. Schwab, and R.D. Theriault. “Managerial Perceptions

Regarding Salary Increase Criteria.” Personnel Psychology, Vol. 29 (1976),

pp. 233-242.

Egli, C.P. “Judicial Refinement of Statistical Evidence in Title VII Cases.”

Connecticut Law Review, Vol. 13 (1981), pp. 515-548.

Ehrenberg, R.G., ed. Do Compensation Policies Matter? Ithaca, NY: ILR

Press, 1990.

Eldred, C. A Report of a Study of Women Training or Working in Outside

Plant Craft Jobs in the Bell System. Westate, Inc., Rockville, MD: 1975.

Elizur, D. Job Evaluation: A Systematic Approach. Hants, England: Gower,

1980.

Elizur, D. Systematic Job Evaluation and Comparable Worth. Brooksville,

VA: Gower Publishing Company, 1987.

Elizur, D. “Systematic Selection of Job Evaluation Items.” Applied

Psychology An International Review, Vol. 36 (1987), pp. 51-59.

Elizur, D. “The Scaling Method of Job Evaluation.” Compensation Review,

Vol. 10 (1978), pp. 34-46.

Elizur, D., and H. Thierry. “Job Evaluation, Comparable Worth, and

Compensation: An Introduction.” Applied Psychology: An International

Review, Vol. 36 (1987), pp. 3-7.

Page 86 GAO/GGD-96-20 Federal Job Classification
Bibliography

Ellerman, D.A., and E.R. Smith. “Generalized and Individual Bias in the

Evaluation of the Work of Women: Sexism in Australia.” Australian

Journal of Psychology, Vol. 35 (1983), pp. 71-79.

Employment Attributes of Recent Science and Engineering Graduates.

National Science Foundation, Publication 80-311. Washington, D.C.: 1980.

England, P. Comparable worth: Theories and Evidence. New York: Aldine

de Gruyter, 1992.

England, P., M. Chassie, and L. McCormack. “Skill Demands and Earnings

in Female and Male Occupations.” Sociology and Social Research, Vol. 66

(1982), pp. 147-168.

England, P., and D. Dunn. “Evaluating Work and Comparable Worth,”

Annual Review of Sociology. Palo Alto, CA: 1988.

England, P., et al. “Explaining Occupational Sex Segregation and Wages:

Findings From a Model With Fixed Effects.” American Sociological

Review, Vol. 53 (1988), pp. 544-558.

England, P. and B.S. Kilbourne. “Using Job Evaluation to Achieve Pay

Equity.” International Journal of Public Administration, Vol. 14 (1991), pp.

823-843.

England, P., and S.D. McLaughlin. “Sex Segregation of Jobs and

Male-female Income Differentials,” Discrimination in Organizations: Using

Social Indicators to Manage Social Change. San Francisco: Jossey-Bass,

1979.

Epperson, L.L. “The Dynamics of Factor Comparison/point Evaluation.”

Public Personnel Management, Vol. 4 (1975), pp. 38-48.

Equal Employment Opportunity Commission. “The Equal Pay Act:

Interpretations.” Federal Register, Vol. 46 (1991), pp. 43848-43852.

Ervin, D., B.J. Thomas, and Zey-Ferrell. “Sex Discrimination and Rewards

in a Public Comprehensive University.” Human Relations, Vol. 37 (1984),

pp. 1005-1025.

Page 87 GAO/GGD-96-20 Federal Job Classification
Bibliography

Evans, S.M., and B.J. Nelson. Wage Justice: Comparable Worth and the

Paradox of Technocratic Reform. Chicago: The University of Chicago

Press, 1989.

Executive Summary of the Classification Modernization Study. State of

Ohio, Ohio’s Bureau of Employment Services, Women’s Division.

Columbus, OH: 1986.

Eyde, L.D. “Evaluating Job Evaluation: Emerging Research Issues for

Comparable Worth Analysis.” Public Personnel Management, Vol. 12

(1983), pp. 425-444.

Fagerlind, I. Formal Education and Adult Earnings: A Longitudinal Study

on the Economic Benefits of Education. Stockholm, Sweden: Almquist and

Wiksell International, 1975.

Farley, L. Sexual Shakedown: The Sexual Harassment of Women on the

Job. New York: Warner Books, 1980.

Feldberg, R.L. “Comparable Worth: Toward Theory and Practice in the

United States.” Signs: Journal of Women in Culture and Society, Vol. 10

(1984), pp. 311-328.

Feldberg, R.L. “Comparable Worth: The Relationship of Method and

Politics,” Ingredients for Women’s Employment Policy. Albany: State

University of New York Press, 1987.

Feldstein, H.R. “Sex-based Wage Discrimination Claims After County of

Washington v. Gunther.” Columbia Law Review, Vol. 81 (1981), pp.

1333-1347.

Ferber, M.A., and B.G. Burnbaum. “Labor Force Participation Patterns and

Earnings of Women Clerical Workers.” The Journal of Human Resources,

Vol. 16 (1981), pp. 416-426.

Ferber, M.A., and W.W. McMahon. “Women’s Expected Earnings and Their

Investment in Higher Education.” The Journal of Human Resources, Vol.

14 (1979), pp. 405-520.

Ferraro, G.A. “Bridging the Wage Gap: Pay Equity and Job Evaluations.”

American Psychologist, Vol. 39 (1984), pp. 1166-1170.

Page 88 GAO/GGD-96-20 Federal Job Classification
Bibliography

Ferris, G.R., and D.C. Gilmore. “A Methodological Note on Job Complexity

Indexes.” Journal of Applied Psychology, Vol. 70 (1985), pp. 225-227.

FES, Factor Evaluation System, Position Classification Standards, General

Introduction, Background, and Instructions, Section VII, Instruction for

the Factor Evaluation System. U.S. Civil Service Commission, Transmittal

Sheet 27. Washington, D.C.: 1977.

Filer, R.K. “Sexual Differences in Earnings: The Role of Individual

Personalities and Tastes.” Human Resources, Vol. 18 (1983), pp. 82-99.

Final Report and Recommendations of the Task Force on State

Compensation and Classification Equity to the 63rd Legislative Assembly.

Task Force on State Compensation and Classification Equity, State of

Oregon. Salem, OR: 1985.

Final Report: PAQ Gender/race Analyses for Arlington County, VA.

Jeanneret and Associates, Inc. Houston, TX: 1986.

Findley, S. “Making Sense of Pay Equity: Issues for a Feminist Political

Practice,” Just Wages: A Feminist Assessment of Pay Equity. Toronto:

University of Toronto Press, 1991.

Finkelstein, J.A., and C.H. Hatch. “Job Evaluation: New Technology, New

Role for HR Managers.” Personnel, Vol. 64 (1987), pp. 5-10.

Fisher, G.D. “Salary Surveys—An Antitrust Prospective.” Personnel

Administrator, (1985) pp. 87-97, 154.

Fitzpatrick, B. “An Objective Test of Job Evaluation Validity.” Personnel,

Vol. 28 (1949), pp. 128-131.

Flammang, J.A. “Effective Implementation: The Case of the Comparable

Worth in San Jose.” Policy Studies Review, Vol. 5 (1986), pp. 815-837.

Flammang, J.A. “Women Made a Difference: Comparable Worth in San

Jose,” The Women’s Movements of the United States and Western Europe.

Philadelphia: Temple University Press, 1987.

Fleishman, E.A. Rating Scale Booklet F-JAS: Fleishman Job Analysis

Survey. Palo Alto, CA: Consulting Psychologists Press, 1992.

Page 89 GAO/GGD-96-20 Federal Job Classification
Bibliography

Fleishman, E.A., and M.D. Mumford. “Evaluating Classifications of Job

Behavior: A Construct Validation of the Ability Requirement Scales.”

Personnel Psychology, Vol. 41 (1991), pp. 523-575.

Fleishman, E.A., and M.E. Reilly. Administrator’s Guide: Fleishman Job

Analysis Survey. Palo Alto, CA: Consulting Psychologists Press, 1992.

Fleishman, E.A., and M.E. Reilly. Handbook of Human Abilities:

Definitions, Measurements, and Job Task Requirements. Palo Alto, CA:

Consulting Psychologist Press, 1992.

Forgionne, G.A., and V.E. Peeters. “Differences in Job Motivation and

Satisfaction Among Female and Male Managers.” Human Relations, Vol. 35

(1982), pp. 101-118.

Fottler, M.D., and T. Bain. “Managerial Aspirations of High School Seniors:

A Comparison of Males and Females.” Journal of Vocational Behavior, Vol.

16 (1980), pp. 83-95.

Fox, M.F. “Sex, Salary, and Achievement: Reward Dualism in Academia.”

Sociology of Education, Vol. 54 (1981), pp. 71-84.

Fox, M.F. “Sex Segregation and Salary Structure in Academics.” Sociology

of Work and Occupations, Vol. 8 (1981), pp. 36-60.

Frank, M.S. “Position Classification: A State-of-the-Art Review and

Analysis.” Public Personnel Management Journal, Vol. 11 (1982), pp.

239-247.

Frank, R.H. “Are Workers Paid Their Marginal Products?” The American

Economic Review, Vol. 74 (1984), pp. 549-571.

Fraser, S.L., S.F. Cronshaw, and R.A. Alexander. “Generalizability Analysis

of a Point Method Job Evaluation Instrument: A Field Study.” Journal of

Applied Psychology, Vol. 69 (1984), pp. 643-647.

Frasher, J.M., R.S. Frasher, and F.B. Wims. “Sex-role Stereotyping in

School Superintendents’ Personnel Decisions.” Sex Roles, Vol. 8 (1982),

pp. 261-268.

Freed, M.G., and D.D. Polsby. “Comparable Worth in the Equal Pay Act.”

University of Chicago Law Review, Vol. 51 (1984), pp. 1078-1111.

Page 90 GAO/GGD-96-20 Federal Job Classification
Bibliography

Freedman, S.M. “The Effects of Subordinate Sex, Pay Equity, and Strength

of Demand on Compensation Decisions.” Sex Roles, Vol. 5 (1979), pp.

649-658.

Freeman, R.B. “Overinvestment in College Training?” The Journal of

Human Behavior, Vol. 10 (1975), pp. 287-311.

Friedman, L., and R.J. Harvey. “Can Raters With Reduced Job Descriptive

Information Provide Accurate Position Analysis Questionnaire (PAQ)

Ratings?” Personnel Psychology, Vol. 39 (1986), pp. 779-796.

Frug, M.J. Women and the Law. Westbury, NY: The Foundation Press,

1992.

Fuchs, V.R. Women’s Quest for Economic Equality. Cambridge, MA:

Harvard University Press, 1988.

Fudge, J., and P. McDermott, eds. Just Wages: A Feminist Assessment of

Pay Equity. Toronto: University of Toronto Press, 1991.

Fulghum, J.B. “The Employer’s Liabilities Under Comparable Worth.”

Personnel Journal, Vol. 62 (1983), pp. 400-412.

Fulghum, J.B. “The Newest Balancing Act: A Comparable Worth Study.”

Personnel Journal, Vol. 63 (1984), pp. 32-38.

Gasaway, L.N. “Comparable Worth: A Post-Gunther Overview.”

Georgetown Law Journal, Vol. 69 (1981), pp. 1123-1169.

Gaskell, J. “What Counts as Skill? Reflections on Pay Equity,” Just Wages:

A Feminist Assessment of Pay Equity. Toronto: University of Toronto

Press, 1991.

Gaskell, J.S. “Conceptions of Skill and the Work of Women: Some

Historical and Political Issues.” The Politics of Diversity: Feminism,

Marxism and Nationalism. London: Verso, 1986.

Gerbner, G., et al., eds. The Analysis of Communication Content:

Developments in Scientific Theories and Computer Techniques.

Huntington, NY: Robert E. Krieger Publishing Company, 1978.

Page 91 GAO/GGD-96-20 Federal Job Classification
Bibliography

Gerhart, B. “Gender Differences in Current and Starting Salaries: The Role

of Performance, College Major, and Job Title.” Industrial and Labor

Relations Review, Vol. 43 (1990), pp. 418-433.

Gerhart, B. and N.E. Cheikh. “Earnings and Percentage Female: A

Longitudinal Study.” Industrial Relations, Vol. 30 (1991), pp. 62-78.

Gerhart, B.A., and G.T. Milklavich. “Salaries, Salary Growth, Promotions of

Men and Women in a Large, Private Firm,” Pay Equity: Empirical Inquiries.

Washington, D.C.: National Academy Press, 1989.

Gethman, B.R. “The Job Market, Sex Bias, and Comparable Worth.” Public

Personnel Management, Vol. 16 (1987), pp. 173-180.

Ghiselli, E.E., J.P. Campbell, and S. Zedeck. Measurement Theory for the

Behavioral Sciences. San Francisco: W.H. Freeman and Company, 1981.

Gibson, J.W., and E.P. Prien. “Validation of Minimum Qualifications.”

Public Personnel Management, Vol. 6 (1977), pp. 447-451.

Giese, S.L., R.A. Alexander, and G.V. Barrett. “Comparison of Students and

Professionals as Subjects in Job Evaluation Research.” Presented at the

4th annual conference of the Society for Industrial and Organizational

Psychology, Boston: 1989.

Giles, B.A., and G.V. Barrett. “Utility of Merit Increases.” Journal of

Applied Psychology, Vol. 55 (1971), pp. 103-109.

Gilligan, C. “In a Different Voice: Women’s Conceptions of Self and

Morality.” Harvard Educational Review, Vol. 47 (1977), pp. 481-517.

Goals and Techniques for a Merit Pay System: Information for State and

Local Government. U.S. Office of Personnel Management, Office of

Intergovernmental Personnel Programs. Washington, D.C.: 1981.

Gold, M.V. A Dialogue on Comparable Worth. Ithaca, NY: ILR Press, 1983.

Goldin, C. Understanding the Gender Gap: An Economic History of

American Women. New York: Oxford University Press, 1990.

Gollob, H.F. “Detecting Sex Bias in Salaries.” American Psychologist, Vol.

39 (1984), pp. 448-451.

Page 92 GAO/GGD-96-20 Federal Job Classification
Bibliography

Gomberg, W. “A Trade Unionist Looks at Job Evaluation.” Journal of

Applied Psychology, Vol. 35 (1951), pp. 1-7.

Gomez-Megia, L.R., ed. Compensation and Benefits. Washington, D.C.:

Bureau of National Affairs Books, 1989.

Gomez-Megia, L.R., R.C. Page, and W.W. Tornow. “A Comparison of the

Practical Utility of Traditional, Statistical, and Hybrid Job Evaluation

Approaches.” Academy of Management Journal, Vol. 25 (1982), pp.

790-809.

Gordon, M.E., and W.J. Fitzgibbons. “Empirical Test of the Validity of

Seniority as a Factor in Staffing Decisions.” Journal of Applied

Psychology, Vol. 67 (1982), pp. 311-319.

Government of Canada Position Information Questionnaire. Canadian

Human Rights Commission, Equal Pay for Work of Equal Value Study Joint

Union/management Initiative. Ottawa: 1987.

Graddick, M.M., and J.L. Farr. “Professionals in Scientific Disciplines:

Sex-related Differences in Working Life Commitments.” Journal of Applied

Psychology, Vol. 68 (1983), pp. 641-645.

Graham, M., and A.C. Hyde. “Comparable Worth in the United States: Legal

and Administrative Developments in the 1980s.” International Journal of

Public Administration, Vol. 14 (1991), pp. 799-821.

Graham-Moore, B., and T.L. Roos. Gainsharing: Plans for Improving

Performance. Washington, D.C.: The Bureau of National Affairs, 1990.

Grams, R., and D.P. Schwab. “An Investigation of Systematic

Gender-related Error in Job Evaluation.” Academy of Management

Journal, Vol. 28 (1985), pp. 279-290.

Grant, D.L. “An Analysis of a Point Rating Job Evaluation Plan.” Journal of

Applied Psychology, Vol. 35 (1951), pp. 236-240.

Gray, J.S. “Custom Made Systems of Job Evaluations.” Journal of Applied

Psychology, Vol. 34 (1950), pp. 378-380.

Gray, J.S. “Adjusting Base Weights in Job Evaluation.” Journal of Applied

Psychology, Vol. 35 (1951), pp. 8-10.

Page 93 GAO/GGD-96-20 Federal Job Classification
Bibliography

Gray, J.S., and M.C. Jones. “Ready Made Versus Custom Made Systems of

Job Evaluation.” Journal of Applied Psychology, Vol. 35 (1951), pp. 11-14.

Gray, M.W. “Legal Perspectives on Sex Equity in Faculty Employment.”

Journal of Social Issues, Vol. 41 (1985), pp. 121-134.

Gray, M.W., and W.L. Scott. “A ‘Statistical’ Remedy for Statistically

Identified Discrimination.” Academe, Vol. 66 (1980), pp. 174-181.

Green, S.B., and T. Stutzman. “An Evaluation of Methods to Select

Respondents for Structured Job-analysis Questionnaires.” Personnel

Psychology, Vol. 39 (1986), pp. 543-564.

Greenberger, E., and L.D. Steinberg. “Sex Differences in Early Labor Force

Experience: Harbinger of Things to Come.” Social Forces, Vol. 62 (1983),

pp. 467-486.

Greig, J.J., P.F. Orazem, and J.P. Mattila. “Measurement Error in

Comparable Worth Pay Analysis: Causes, Consequences, and Corrections.”

Journal of Social Issues, Vol. 45 (1989), pp. 135-151.

Grider, D., and L.A. Toombs. “Disproving Valuation Discrimination: A

Study of Evaluator Gender Bias.” ACA Journal, Vol. 2 (1993), pp. 24-33.

Gronau, R. “Sex-related Wage Differentials and Women’s Interrupted

Labor Careers—The Chicken or the Egg.” Journal of Labor Economics,

Vol. 6 (1988), pp. 277-301.

Gross, A.L., J. Faggen, and K. McCarthy. “The Differential Predictability of

the College Performance of Males and Females.” Educational and

Psychological Measurement, Vol. 34 (1974), pp. 363-365.

Grubb, W.N., and R.H. Wilson. “Sources of Increasing Inequality in Wages

and Salaries, 1960-1980.” Monthly Labor Review, Vol. 112 (1989), pp. 3-13.

Grune, J.A., ed. Manual on Pay Equity: Raising Wages for Women’s Work.

Washington, D.C.: Committee on Pay Equity, 1979.

Grune, J.A., and N. Reder. “Pay Equity: An Innovative Public Policy

Approach to Eliminating Sex-based Wage Discrimination.” Public

Personnel Management Journal, Vol. 12 (1984), pp. 70-80.

Page 94 GAO/GGD-96-20 Federal Job Classification
Bibliography

Grune, J.A., and N. Reder. “Addendum—Pay Equity: An Innovative Public

Policy Approach to Eliminating Sex-based Wage Discrimination.” Public

Personnel Management Journal, Vol. 13 (1984), pp. 70-80.

Grusky, O. “Career Mobility and Organizational Commitment.”

Administrative Science Quarterly, Vol. 10 (1966), pp. 488-502.

Guion, R.H. “A Parametric Study of Comparable Worth: Social Judgment

Theory and Latent Trait Theory Applied to Job Evaluation.” Abstract, 1981.

Gunderson, M. “Probit and Logit Estimates of Labor Force Participation.”

Industrial Relations, Vol. 19 (1980), pp. 216-220.

Gupta, N., and G.D. Jenkins, Jr. “Practical Problems in Using Job

Evaluation Systems to Determine Compensation.” Human Resource

Management Review, Vol. 1 (1991), pp. 133-144.

Gutek, B.A., and L. Larwood. “Introduction: Women’s Careers Are

Important and Different,” Women’s Career Development. Newbury Park,

CA: Sage, 1987.

Haberfeld, Y. “Employment Discrimination: An Organizational Model.”

Academy of Management Journal, Vol. 35 (1992), pp. 161-180.

Hahn, D.C., and R.L. Dipboye. “Effects of Training and Information on the

Accuracy and Reliability of Job Evaluations.” Journal of Applied

Psychology, Vol. 73 (1988), pp. 146-153.

Haignere, L.V., C.A. Chertos, and R.J. Steinberg. Managerial Promotion in

the Public Sector: The Importance of Eligibility Requirements on Women

and Minorities. Albany, NY Center for Women in Government, State

University of New York. Albany: 1981.

Hall, J.K., and P.E. Spector. “Relationships of Work Stress Measures for

Employees with the Same Job.” Work and Stress, Vol. 5 (1991), pp. 29-35.

Hall, R., and G.V. Barrett. “Payday for Patients: Federal Guidelines or a

Job-sample Approach?” American Psychologist, Vol. 46 (1977), pp. 586-588.

Hallard, A.H., and H.G. Schultz. “A Factor Analysis of a Salary Job

Evaluation Plan.” Journal of Applied Psychology, Vol. 35 (1952), pp.

243-246.

Page 95 GAO/GGD-96-20 Federal Job Classification
Bibliography

Handbook for Analyzing Jobs. U.S. Department of Labor. Washington,

D.C.: 1972.

Handbook of Occupational Groups and Series. U.S. Office of Personnel

Management, Washington, D.C.: 1990.

Hansen, R.D., and V.E. O’Leary. “Sex-Determined Attributions,” Women,

Gender, and Social Psychology. Hillsdale, NJ: Lawrence Erlbaum

Associates, 1985.

Harding, F.D., J.M. Madden, and K. Colson. “Analysis of a Job Evaluation

System.” Journal of Applied Psychology, Vol. 44 (1960), pp. 354-357.

Harding, S. The Science Question in Feminism. Ithica, NY: Cornell

University Press, 1986.

Harrel, M.S., et al. “Predicting Compensation Among MBA Graduates Five

and Ten Years After Graduation.” Journal of Applied Psychology, Vol. 62

(1971), pp. 636-640.

Hartenian, L.S., and N.B. Johnson. “Establishing the Reliability and Validity

of Wage Surveys.” Public Personnel Management, Vol. 20 (1991), pp. 367-

383.

Hartman, A. “Comparable Worth.” Harvard Women’s Law Journal, Vol. 6

(1983), pp. 201-218.

Hartmann, H.I. “Capitalism, Patriarchy, and Job Segregation by Sex,”

Women and the Workplace. Chicago: University of Chicago Press, 1976.

Hartmann, H.I. “The Family as the Locus of Gender, Class, and Political

Struggle: The Example of Housework.” Journal of Women in Culture and

Society, Vol. 6 (1981), pp. 366-394.

Hartmann, H.I. “Comparable Worth and Women’s Economic

Independence,” Ingredients for Women’s Employment Policy. Albany:

State University of New York Press, 1987.

Hartmann, H.I., ed. Comparable Worth: New Directions for Research.

Washington, D.C.: National Academy Press, 1985.

Page 96 GAO/GGD-96-20 Federal Job Classification
Bibliography

Hartmann, H.I., ed. Computer Chips and Paper Clips: Technology and

Women’s Employment Case Studies and Policy Perspectives, Vol. II.

Washington, D.C.: National Academy Press, 1986.

Hartmann, H.I. “Internal Labor Markets and Gender: A Case Study of

Promotions,” Gender in the Workplace. Washington, D.C.: The Brookings

Institution, 1987.

Hartmann, H.I. “Capitalism, Patriarchy, and Job Segregation by Sex,”

Women, Class, and the Feminist Imagination: A Socialist-Feminist Reader.

Philadelphia: Temple University Press, 1990.

Hartmann, H.I., R.E. Krant, and L.A. Tilly, eds. Computer Chips and Paper

Clips: Technology and Women’s Employment, Vol. I. Washington, D.C.:

National Academy Press, 1986.

Hartmann, H.I., and D.M. Pearce. High Skill and Low Pay: The Economics

of Child Care at Work. Institute for Women’s Policy Research. Washington,

D.C.: 1989.

Hartmann, H., and R. Spalter-Roth. Women in Telecommunications: An

Acception to the Rule. Institute for Women’s Policy Research. Washington,

D.C.: 1989.

Hartog, J. “On the Multicapability Theory of Income Distribution.”

European Economic Review, Vol. 19 (1977), pp. 157-171.

Hartog, J. “Earnings and Capability Requirements.” Review of Economics

and Statistics, Vol. 62 (1980), pp. 230-240.

Harvey, R.J. “Job Analysis.” Handbook of Industrial and Organizational

Psychology, 2nd ed., Vol. II. Palo Alto, CA: Consulting Psychologists Press,

Inc., 1991.

Harvey, R.J., et al. “Dimensionality of the Job Element Inventory, a

Simplified Worker-oriented Job Analysis Questionnaire.” Journal of

Applied Psychology, Vol. 73 (1988), pp. 639-646.

Hay, E.N. “Planning for Fair Salaries and Wages.” Personnel Journal, Vol.

18 (1939), pp. 141-150.

Page 97 GAO/GGD-96-20 Federal Job Classification
Bibliography

Hay, E.N. “Job Evaluation—A Discussion.” Personnel Journal, Vol. 28

(1949), pp. 262-266.

Hay, E.N. “Techniques of Securing Agreement in Job Evaluation

Committees.” Personnel, Vol. 26 (1950), pp. 307-312.

Hay, E.N. “The Application of Weber’s Law to Job Evaluation Estimates.”

Journal of Applied Psychology, Vol. 34 (1950), pp. 102-104.

Hay, E.N. “The Attitude of the American Federation of Labor on Job

Evaluation.” Personnel Journal, Vol. 26, pp. 163-169.

Hay, E.N. “What Kind of Job Evaluation?—A Reply.” Public Personnel

Review, Vol. 14 (1953), pp. 123-127.

Hay, E.N., and D. Purves. “The Profile Method of High-level Job

Evaluation.” Personnel, Vol. 28 (1951), pp. 162-170.

Hay, E.N., and D. Purves. “A New Method of Job Evaluation: A Guide

Chart-profile Method.” Personnel, Vol. 31 (1954), pp. 72-80.

Hayes, R. “Gender Nontraditional or Sex Atypical or Gender Dominant or

... Research: Are We Measuring the Same Thing?” Journal of Vocational

Behavior, Vol. 29 (1986), pp. 79-88.

Hazel, J.T. “Reliability of Job Ratings as a Function of Time Spent on

Evaluation.” The Journal of Industrial Psychology, Vol. 4 (1966), pp. 16-19.

Hazel, J.T., J.M. Madden, and E.E. Christal. “Agreement Between

Worker-supervisor Descriptions of the Worker’s Job.” Journal of Industrial

Psychology, Vol. 9 (1964), pp. 71-79.

Hegtvedt, K.A. “Fairness Conceptualizations and Comparable Worth.”

Journal of Social Issues, Vol. 45 (1989), pp. 81-97.

Heilman, M.E. “High School Students Occupational Interest as a Function

of Projected Sex Ratios in Male Dominated Occupations.” Journal of

Applied Psychology, Vol. 64 (1979), pp. 275-279.

Henderson, R. Compensation Management, 5th ed. Englewood Cliffs, NJ:

Prentice Hall, 1989.

Page 98 GAO/GGD-96-20 Federal Job Classification
Bibliography

Henderson, R.I., and K.L. Clarke. “Job Pay for Job Worth: Designing and

Managing an Equitable Job Classification and Pay System.” Research

Monograph 86, College of Business Administration. Atlanta: Georgia State

University Business Publishing Division, 1981.

Hennig, M., and A. Jardim. The Managerial Woman. New York: Simon and

Schuster, 1977.

Hermkens, P., and P. Van Wijngaarden. “Job Evaluation and Justification

Criteria for Income Differentials.” Applied Psychology: An International

Review, Vol. 36 (1987), pp. 109-117.

Hersch, J. “Male-female Differences in Hourly Wages: The Role of Human

Capital, Working Conditions and Housework.” Industrial and Labor

Relations Review, Vol. 44 (1991), pp. 746-759.

Hewitt, B.M., and R.D. Goldman. “Occam’s Razor Slices Through the Myth

That College Women Overachieve.” Journal of Educational Psychology,

Vol. 67 (1975), pp. 325-330.

Hill, M.A., and M.R. Killingsworth, eds. Comparable Worth: Analysis and

Evidence. Ithaca, NY: ILR Press, 1989.

Hitt, M.A., and S.H. Barr. “Managerial Selection Decision Models: The

Examination of Configural Cue Processing.” Journal of Applied

Psychology, Vol. 74 (1989), pp. 53-61.

Hoffmann, C.C., and K.P. Hoffmann. “Does Comparable Worth Obscure the

Real Issues?” Personnel Journal, Vol. 66 (1987), pp. 83-95.

Hoffmann, C., and S.S. Reed. “Sex Discrimination?—The XYZ Affair.” The

Public Interest, Vol. 62 (1981), pp. 21-39.

Hogan, J.C., et al. “Reliability and Validity of Methods for Evaluating

Perceived Physical Effort.” Journal of Applied Psychology, Vol. 65 (1980),

pp. 672-679.

Hollenbeck, J.R., et al. “Sex Differences in Occupational Choice, Pay, and

Worth: A Supply-side Approach to Understanding the Male-female Wage

Gap.” Personnel Psychology, Vol. 40 (1987), pp. 715-743.

Page 99 GAO/GGD-96-20 Federal Job Classification
Bibliography

Holsti, O.R. Content Analysis for the Social Sciences and Humanities.

Reading, MA: Addison-Wesley Publishing Company, 1969.

Hornsby, J.S., P.G. Benson, and B.N. Smith. “An Investigation of Gender

Bias in the Job Evaluation Process.” Journal of Business and Psychology,

Vol. 2 (1987), pp. 150-159.

Horrigan, J., and A. Harriman. “Comparable Worth: Public Sector Unions

and Employers Provide a Model for Implementing Pay Equity.” Labor Law

Journal, Vol. 39 (1988), pp. 704-711.

How to Write Position Descriptions. U.S. Civil Service Commission.

Washington D.C.: 1978.

Huber, V.L. “Comparison of Supervisor-incumbent and Female-male

Multidimensional Job Evaluation Ratings.” The Journal of Applied

Psychology, Vol. 76 (1991), pp. 115-121.

Hunt, T. “Futurism and Futurists in Personnel.” Public Personnel

Management, Vol. 13 (1984), pp. 511-520.

Hunter, L. “A Method for Monitoring University Faculty Salary Policies for

Sex Bias,” Salary Equity: Detecting Sex Bias in Salaries Among College and

University Professors. Lexington, MA: Lexington Books, 1979.

Huseman, R.C., J.D. Hatfield, and E.W. Miles. “Test for Individual

Perceptions of Job Equity: Some Preliminary Findings.” Perceptual and

Motor Skills, Vol. 61 (1985), pp. 1055-1064.

Hutner, F.C. Equal Pay for Comparable Worth: The Working Woman’s

Issue of the Eighties. New York: Praeger, 1986.

Instructions for the Factor Evaluation System. U.S. Civil Service

Commission. Washington, D.C.: 1977.

Interpretative Bulletin of Code of Federal Regulations, Equal Pay for Equal

Work, Under the Fair Labor Standards Act, Title 29 Part 800. U.S.

Department of Labor, Employment Standards Administration, Wage and

Hour Division. Washington, D.C.: 1971.

Introductory Guide for Use With the Position Analysis Questionnaire. PAQ

Services, Inc. Logan, UT: 1991.

Page 100 GAO/GGD-96-20 Federal Job Classification
Bibliography

Ivancevich, J.M., and S.M. Smith. “Job Difficulty as Interpreted by

Incumbents: A Study of Nurses and Engineers.” Human Relations, Vol. 35

(1982), pp. 391-412.

Jackson, L.A. “Relative Deprivation and the Gender Wage Gap.” Journal of

Social Issues, Vol. 45 (1989), pp. 117-133.

Jackson, L.A., and S.V. Grabski. “Perceptions of Fair Pay and the Gender

Wage Gap.” Journal of Applied Social Psychology, Vol. 18 (1988), pp.

606-625.

Jacobs, J.A. Revolving Doors: Sex Segregation and Women’s Careers.

Stanford, CA: Stanford University Press, 1989.

Jacques, E. Time-span Handbook. London: Heinemann, 1971.

Jacques, E. “Taking Time Seriously in Evaluating Jobs.” Harvard Business

Review, Vol. 57 (1979), pp. 124-132.

Jagacinski, C.M., W.K. LeBold, and K.W. Linden. “The Relative Career

Advancement of Men and Women Engineers in the United States.” Work

and Stress, Vol. 1 (1987), pp. 235-247.

James, L.R. “Aggregation Bias in Estimates of Perceptual Agreement.”

Journal of Applied Psychology, Vol. 67 (1982), pp. 219-229.

Jaussaud, D.P. “Can Job Evaluation Systems Help Determine the

Comparable Worth of Male and Female Occupations?” Journal of

Economic Issues, Vol. 18 (1984), pp. 473-482.

Jeanneret, P.R. “Equitable Job Evaluation and Classification With the

Position Analysis Questionnaire.” Compensation Review, Vol. 12 (1980),

pp. 32-47.

Jeanneret, P.R. “Affidavit of Paul Richard Jeanneret.” Houston, TX: 1983.

Job Evaluation: A Practical Guide. British Institute of Management.

Southampton: 1961.

Job Evaluation Handbook. State of Iowa. Des Moines.

Page 101 GAO/GGD-96-20 Federal Job Classification
Bibliography

Job Evaluation. International Labour Organization. Geneva, Switzerland:

1986.

Johnson, N.B. and R.A. Ash. “Integrating the Labor Market With Job

Evaluation: Clearing the Cobwebs.” Presented at the 1st Annual Society

for Industrial and Organizational Psychology Conference. Chicago: 1986.

Jones, M.B., C.A. Braddick, and P.M. Shafer. “Will Broadband Replace

Traditional Salary Structures?” Journal of Compensation and Benefits, Vol.

7 (1991), pp. 30-35.

Kahn, A., R.E. Nelson, and W.P. Gaeddert. “Sex of Subject and Sex

Composition of the Group as Determinants of Reward Allocations.”

Journal of Personality and Social Psychology, Vol. 38 (1980), pp. 737-750.

Kahn, A., et al. “Equity and Equality: Male and Female Means to a Just

End.” Basic and Applied Social Psychology, Vol. 1 (1980), pp. 173-197.

Kahn, A.S., and W.P. Gaeddert. “From Theories of Equity to Theories of

Justice: The Liberating Consequences of Studying Women,” Women,

Gender, and Social Psychology. Hillsdale, NJ: Lawrence Erlbaum

Associates, 1985.

Kalin, R., and D.C. Hodgins. “Sex Bias in Judgements of Occupational

Suitability.” Canadian Journal of Behavior Science, Vol. 16 (1984), pp.

311-325.

Kandel, T. What Women Earn. New York: The Linden Press/Simon and

Schuster, 1981.

Kandel, W.L. “Current Developments in EEO: Pregnancy Discrimination in

Context.” Employee Relations Law Journal, Vol. 5 (1979), pp. 258-268.

Kanter, R.M. Men and Women of the Corporation. New York: Basic Books,

1977.

Kanter, R.M. “The Impact of Hierarchical Structures on the Work Behavior

of Women and Men.” Social Problems, Vol. 23 (1978), pp. 415-430.

Kanter, R.M. “From Status to Contribution: Some Organizational

Implications of the Changing Basis for Pay.” Personnel, Vol. 64 (1987), pp.

12-37.

Page 102 GAO/GGD-96-20 Federal Job Classification
Bibliography

Katzell, M.E., and W.C. Byham, eds. Women in the Work Force:

Confrontation With Change. New York: Behavioral Publication Inc., 1972.

Kauggman, B.E., ed. How Labor Markets Work. Lexington, MA: Lexington

Books, 1988.

Kaye, D. “Statistical Evidence of Discrimination.” Journal of the American

Statistical Association, Vol. 77 (1982), pp. 773-792.

Keith, P.M. “Sex, Occupation, Year of Graduation and Perceptions of Job

Factors.” Journal of Employment Counseling, Vol. 15 (1980), pp. 180-186.

Kelley, H.H. “The Processes of Causal Attribution.” American Psychologist,

Vol. 28 (1973), pp. 107-128.

Kelly, R.M., and J. Bayes, eds. Comparable Worth, Pay Equity, and Public

Policy. New York: Greenwood Press, 1989.

Kerr, C., and L.H. Fisher. “Effect of Environment and Administration on

Job Evaluation.” Harvard Business Review, Vol. 28 (1950), pp. 77-96.

Kessler-Harris, A. “The Debate Over Equality for Women in the Work

Place: Recognizing Differences,” Women and Work 1: An Annual Review.

Beverly Hills, CA: Sage Publications, Inc., 1985.

Kessler-Harris, A. “The Just Price, the Free Market, and the Value of

Women.” Feminist Studies, Vol. 14 (1988), pp. 235-250.

Kessler-Harris, A. A Women’s Wage: Historical Meanings and Social

Consequences. Lexington, KY: The University Press of Kentucky, 1990.

Kiesler, S., and T. Finholt. “The Mystery of RSI.” American Psychologist,

Vol. 43 (1988), pp. 1004-1015.

Kilberg, W.J. “The Earnings Gap and Comparable Worth.” Employee

Relations Law Journal, Vol. 102 (1985), pp. 579-583.

Killingsworth, M.R. “Comparable Worth in the Job Market: Estimating Its

Effects.” Monthly Labor Review, Vol. 108 (1985), pp. 39-41.

Killingsworth, M.R. The Economics of Comparable Worth. Kalamazoo, MI:

W.E. Upjohn Institute for Employment Research, 1990.

Page 103 GAO/GGD-96-20 Federal Job Classification
Bibliography

Knapp, L., and R.R. Knapp. “Clustered Occupational Interest Measurement

Based on Sex-balanced Inventory Items.” Journal of Educational

Measurement, Vol. 19 (1982), pp. 75-81.

Knepper, J.K. Attitudes Towards Wage-setting in the Private Sector: A Case

Study. The National Commission on Working Women. Washington, D.C.

Komarovsky, M. “Female Freshman View Their Future: Career Salience

and Its Correlates.” Sex Roles, Vol. 8 (1982), pp. 299-314.

Kovach, K.A. “Implicit Stereotyping in Personnel Decisions.” Personnel

Journal, Vol. 60 (1981), pp. 716-722.

Kozlowski, S.W.J., and J.K. Ford. “Rater Information Acquisition

Processes: Tracing the Effects of Prior Knowledge, Performance Level,

Search Constraint, and Memory Demand.” Organizational Behavior and

Human Decision Processes, Vol. 49 (1991), pp. 282-301.

Krefting, L.A., P.K. Berger, and M.J. Wallace. “The Contribution of Sex

Distribution, Job Content, and Occupational Classification to Job

Sextyping: Two Studies.” Journal of Vocational Behavior, Vol. 13 (1978),

pp. 181-191.

Kress, A.L. “Sound Wage Payment Policies,” The AMA Handbook of Wage

and Salary Administration: Tested Compensation Methods for Factory,

Office, and Managerial Personnel. New York: American Management

Association, 1950.

Krippendorff, K. Content Analysis: An Introduction to Its Methodology.

Newbury Park, CA: Sage, 1980.

Kurtz, M., and E.C. Hocking. “Nurses v. Tree-trimmers.” Public Personnel

Management Journal, Vol. 12 (1983), pp. 369-381.

Lacey, N. “Legislation Against Sex Discrimination: Questions From a

Feminist Perspective.” Journal of Law and Society, Vol. 14 (1987), pp.

411-421.

Laking, J., and R. Roark. Retailing Job Analysis and Job Evaluation. New

York: National Retail Merchants Association, 1975.

Page 104 GAO/GGD-96-20 Federal Job Classification
Bibliography

Langstroth, L. “Job Evaluation Discussion.” Personnel Journal, Vol. 29

(1950), pp. 180-182.

Langwell, K.M. “Real Returns to Career Decisions: The Physicians

Specialty and Location Choices.” The Journal of Human Resources, Vol. 15

(1980), pp. 278-86.

Lanham, E. “Job Evaluation in Municipalities.” Public Personnel Review,

Vol. 14 (1953), pp. 26-30.

Lanham, E. “Policies and Practices in Job Evaluation: A Survey.”

Personnel, Vol. 29 (1953), pp. 492-498.

Lanham, E. Job Evaluation. New York: McGraw-Hill Book Co, 1955.

Larwood, L., and U.E. Gattiker. “A Comparison of the Career Paths Used

by Successful Women and Men,” Women’s Career Development. Newbury

Park, CA: Sage, 1987.

Larwood, L., A.H. Stromberg, and B.A. Gutek, eds. Women and Work 1: An

Annual Review. Beverly Hills, CA: Sage Publications, Inc., 1985.

Lawler, E.E., III. Pay and Organizational Effectiveness: A Psychological

View. New York: McGraw-Hill, 1971.

Lawler, E.E., III. Pay and Organization Development. Reading, MA:

Addison-Wesley Publishing Company, 1981.

Lawler, E.E., III. “What’s Wrong With Point-factor Job Evaluation.”

Personnel, Vol. 64 (1987), pp. 38-44.

Lawler, E.E., III. Strategic Pay: Aligning Organizational Strategies and Pay

Systems. San Francisco: Jossey-Bass, 1990.

Lawler, E.E., III, and J.R. Hackman. “Impact of Employee Participation in

the Development of Pay Incentive Plans.” Journal of Applied Psychology,

Vol. 53 (1969), pp. 467-471.

Lawshe, C.H. “Studies in Job Evaluation: The Adequacy of Abbreviated

Point Ratings for Hourly-paid Jobs in Three Industrial Plants.” Journal of

Applied Psychology, Vol. 29 (1945), pp. 177-184.

Page 105 GAO/GGD-96-20 Federal Job Classification
Bibliography

Lawshe, C.H. “Toward Simplified Job Evaluation.” Personnel, Vol. 22

(1945), pp. 153-160.

Lawshe, C.H. “The Reliability of Two Job Evaluation Systems.” American

Psychologist, Vol. 2 (1947), pp. 339.

Lawshe, C.H., and S.L. Alessi. “Studies in Job Evaluation: Analysis of

Another Point Rating Scale for Hourly-paid Jobs and the Adequacy of an

Abbreviated Scale.” Journal of Applied Psychology, Vol. 30 (1946), pp.

310-319.

Lawshe, C.H., and P.C. Fabro. “Studies in Job Evaluation: The Reliability of

an Abbreviated Job Evaluation System.” Journal of Applied Psychology,

Vol. 43 (1949), pp. 158-166.

Lawshe, C.H., and A.A. Maleski. “Studies in Job Evaluation: An Analysis of

Point Ratings for Salary Paid Jobs in an Industrial Plant.” Journal of

Applied Psychology, Vol. 30 (1946), pp. 117-128.

Lawshe, C.H., and E.J. McCormick. “What Do You Buy With the Wage or

Salary Dollar?” Personnel, Vol. 24 (1947), pp. 102-106.

Lawshe, C.H., and G.A. Satter. “Studies in Job Evaluation: Factor Analyses

of Point Ratings for Hourly-paid Jobs in Three Industrial plants.” Journal

of Applied Psychology, Vol. 28 (1944), pp. 189-198.

Lawshe, C.H., and R.F. Wilson “Studies in Job Evaluation: An Analysis of

the Factor Comparison Systems as It Functions in a Paper Mill.” Journal of

Applied Psychology, Vol. 30 (1946), pp. 426-434.

Lawshe, C.H., and R. Wilson. “Studies in Job Evaluation: The Reliability of

Two Point Rating Systems.” Journal of Applied Psychology, Vol. 31 (1947),

pp. 355-365.

Lee, B.A., D.W. Leslie, and S.G. Olswang. “Implications of Comparable

Worth for Academe.” Journal of Higher Education, Vol. 58 (1987), pp.

609-628.

Lee, Y.S. “Shaping Judicial Response to Gender Discrimination in

Employment Compensation.” Public Administration Review, Vol. 49

(1989), pp. 420-430.

Page 106 GAO/GGD-96-20 Federal Job Classification
Bibliography

Leonard, J.S. “Executive Pay and Firm Performance,” Do Compensation

Policies Matter? Ithaca, NY: ILR Press, 1990.

Lester, R.A. Reasoning About Discrimination: The Analysis of Professional

and Executive Work in Federal Antibias Programs. Princeton, NJ:

Princeton University Press, 1980.

Leuptow, L.B. “Sex-typing and Change in the Occupational Choices of High

School Seniors: 1964-1975.” Sociology of Education, Vol. 54 (1981), pp.

16-24.

Levin, M. “Comparable Worth: The Feminist Road to Socialism.”

Commentary, (1983), pp. 13-19.

Levine, E.L., M. Bennett, and R.A. Ash. “Evaluation and Use of Four Job

Analysis Methods for Personnel Selection.” Public Personnel Management,

Vol. 8 (1979), pp. 146-151.

Lewis, C.T. “Assessing the Validity of Job Evaluation.” Public Personnel

Management, Vol. 18 (1989), pp. 45-63.

Lewis, C.T., and C.K. Stevens. “An Analysis of Job Evaluation Committee

and Job Holder Gender Effects on Job Evaluation.” Public Personnel

Management, Vol. 19 (1990), pp. 271-278.

Lewis, G. “Sexual Segregation of Occupations and Earnings Differential in

Federal Employment.” Public Administration Quarterly, Vol. 9 (1985), pp.

274-290.

Lewis, G.B. “Clerical Work and Women’s Earnings in the Federal Civil

Service,” Comparable Worth, Pay Equity, and Public Policy. New York:

Greenwood Press, 1988.

Lindsay, C.M., and M.T. Maloney. “A Model and Some Evidence

Concerning the Influence of Discrimination on Wages.” Economic Inquiry,

Vol. 26 (1988), pp. 645-660.

Linn, R.L. “Fair Test Use in Selection.” Review of Educational Research,

Vol. 43 (1973), pp. 139-161.

Livernash, E.R., ed. Comparable Worth: Issues and Alternatives. Equal

Employment Advisory Council. Washington, D.C.: 1980.

Page 107 GAO/GGD-96-20 Federal Job Classification
Bibliography

Livernash, E.R., ed. Comparable Worth: Issues and Alternatives, 2nd ed.

rev. Equal Employment Advisory Council. Washington, D.C.: 1984.

Livy, B. Job Evaluation: A Critical Review. New York: John Wiley and

Sons, 1975.

Lloyd, C.B., and B.T. Niemi. The Economics of Sex Differentials. New

York: Columbia University Press, 1979.

Lo Bosco, M. “Job Analysis, Job Evaluation, and Job Classification.”

Personnel, Vol. 62 (1985), pp. 70-74.

Locke, N. “Few Factors or Many?—An Analysis of a Point System of

Classification.” Personnel, Vol. 25 (1949), pp. 442-448.

Loeb, J.W., M.A. Ferber, and H.M. Lowry. “The Effectiveness of Affirmative

Action for Women.” Journal of Higher Education, Vol. 49 (1978), pp.

218-230.

London, M. “Employee Perceptions of the Job Reclassification Process.”

Personnel Psychology, Vol. 29 (1976), pp. 67-77.

Long, J.V. “The Idiosyncratic Determiners of Salary Differences,” Salary

Equity: Detecting Sex Bias in Salaries Among College and University

Professors. Lexington, MA: Lexington Books, 1979.

Lorber, L.Z., et al. Sex and Salary: A Legal and Personnel Analysis of

Comparable Worth. The American Society for Personnel Administration.

Alexandria, VA: 1985.

Lott, M.R. Wage Scales and Job Evaluation: Scientific Determination of

Wage Rates on the Basis of Services Rendered. New York: Ronald Press,

1926.

Lowe, R.H., and M.A. Wittig. “Comparable Worth: Individual,

Interpersonal, and Structural Considerations.” Journal of Social Issues,

Vol. 45 (1989), pp. 223-246.

Lumpton, T., ed. Payment Systems: Selected Reading. Baltimore, MD:

Penguin Books, 1972.

Page 108 GAO/GGD-96-20 Federal Job Classification
Bibliography

Lunneborg, P.W. “Service vs. Technical Interest—Biggest Sex Difference

of All?” Vocational Guidance Quarterly, Vol. 28 (1979), pp. 146-153.

Lutes, D., and N. Rothchild. “Compensation: Pay Equity Loses to Chicken

Little and Other Excuses.” Personnel Journal, Vol. 65 (1986), pp. 124-130.

Lutz, L.D. “What Kind of Job Evaluation?” Public Personnel Review, Vol.

14 (1953), pp. 119-122.

Lynn, N.B., and R.E. Vaden. “Toward a Non-sexist Personnel Opportunity

Structure: The Federal Executive Bureaucracy.” Public Personnel

Management, Vol. 8 (1979), pp. 209-215.

Lytle, C.W. Job Evaluation Methods. New York: The Ronald Press, 1946.

Maccoby, E.E. and C.N. Jacklin. The Psychology of Sex Differences.

Stanford, CA: Stanford University Press, 1974.

MacDonald, N.E., and J.S. Hyde. “Fear of Success, Need Achievement, and

Fear of Failure: A Factor Analytic Study.” Sex Roles, Vol. 6 (1980), pp.

695-711.

Madden, J.M. The Methods and Foundations of Job Evaluation in the

United States Air Force. Lackland Air Force Base, Personnel Laboratory,

Aeronautical System Division, Air Force System Command, Technical

Report ASD-TR-61-100. 1961.

Madden, J.M. “The Effect of Varying the Degree of Rater Familiarity in Job

Evaluation.” Personnel Administration, Vol. 25 (1962), pp. 42-46.

Madden, J.M. “A Further Note on the Familiarity Effect in Job Evaluation.”

Personnel Administration, Vol. 26 (1963), pp. 52-53.

Madden, J.M. “Policy-capturing Model for Analyzing Individual and Group

Judgement in Job Evaluation.” Journal of Industrial Psychology, Vol. 2

(1964), pp. 36-42.

Madden, J.M., and R.D. Bourdon. “Effects of Variations in Rating Scale

Formats on Judgment.” Journal of Applied Psychology, Vol. 48 (1964), pp.

147-151.

Page 109 GAO/GGD-96-20 Federal Job Classification
Bibliography

Madigan, R.M. “Comparable Worth Judgments: A Measurement Properties

Analysis.” Journal of Applied Psychology, Vol. 70 (1985), pp. 137-147.

Madigan, R.M., and F.S. Hills. “Job Evaluation and Pay Equity.” Public

Personnel Management, Vol. 17 (1988), pp. 323-330.

Madigan, R.M., and D.J. Hoover. “Effects of Alternative Job Evaluation

Methods on Decisions Involving Pay Equity.” Academy of Management

Journal, Vol. 29 (1986), pp. 84-100.

Maggio, R. The Dictionary of Bias-free Usage: A Guide to

Nondiscriminatory Language. Phoenix: Oryx Press, 1991.

Mahoney, T.A., ed. Compensation and Reward Perspectives. Homewood,

IL: Richard D. Irwin, Inc., 1979.

Mahoney, T.A. “Organizational Hierarchy and Position Worth.” Academy of

Management Journal, Vol. 22 (1979), pp. 726-737.

Mahoney, T.A. “Approaches to the Definition of Comparable Worth.”

Academy of Management Review, Vol. 8 (1983), pp. 14-22.

Mahoney, T.A. “Understanding Comparable Worth: A Societal and Political

Perspective.” Research in Organizational Behavior, Vol. 9 (1987), pp.

209-245.

Mahoney, T.A. “The Symbolic Meaning of Pay Contingencies.” Human

Resource Management Review, Vol. 1 (1991), pp. 179-192.

Mahoney, T.A., and R.H. Blake. “Judgements of Appropriate Occupational

Pay as Influenced by Occupational Characteristics and Sex

Characterizations.” Applied Psychology An International Review, Vol. 36

(1987), pp. 25-38.

Mahoney, T.A., B. Rosen, and S.L. Rynes. “Where Do Compensation

Specialists Stand on Comparable Worth?” Compensation Review, Vol. 16

(1984), pp. 27-40.

Majeres, R.L. “Sex Differences in Symbol-digit Substitution and Speeded

Matching.” Intelligence, Vol. 7 (1983), pp. 313-327.

Page 110 GAO/GGD-96-20 Federal Job Classification
Bibliography

Major, B. “Gender Differences in Comparisons and Entitlement:

Implications for Comparable Worth.” Journal of Social Issues, Vol. 45

(1989), pp. 99-115.

Major, B., and B. Forcey. “Social Comparisons and Pay Evaluations:

Preferences for Same-sex and Same-job Wage Comparisons.” Journal of

Experimental Social Psychology, Vol. 21 (1985), pp. 393-405.

Major, B., and E. Konar. “An Investigation of Sex Differences in Pay

Expectations and Their Possible Causes.” Academy of Management

Journal, Vol. 27 (1984), pp. 777-792.

Makela, C.J. “From Equality to Equity: The Path to Comparable Worth.”

Educational Record, Vol. 66 (1985), pp. 14-18.

Manese, W.R. Occupational Job Evaluation: A Research-based Approach

to Job Classification. New York: Quorum Books, 1988.

Mangum, S.L. “Comparable Worth and Pay Setting in the Public and

Private Sectors.” Journal of Collective Negotiations, Vol. 17 (1988), pp.

1-12.

Marini, M.M., and M.C. Brinton. “Sex Typing in Occupational

Socialization,” Sex Segregation in the Workplace: Trends, Explanations,

Remedies. Washington, D.C.: National Academy Press, 1984.

Martin, J., et al. “Now That I Can Have It, I’m Not So Sure I Want It,”

Women’s Career Development. Newbury Park, CA: Sage, 1987.

Martin, M.P., and J.D. Williams. “Effects of Statewide Salary Equity

Provisions on Institutional Salary Policies: A Regression Analysis,” Salary

Equity: Detecting Sex Bias in Salaries Among College and University

Professors. Lexington, MA: Lexington Books, 1979.

Mathews, J.J., W.E. Collins, and B.B. Cobb. “A Sex Comparison of Reasons

for Attrition in Male-dominated Occupations.” Personnel Psychology, Vol.

27 (1974), pp. 535-541.

Matlin, M.W. The Psychology of Women. New York: Holt, Reinhart, and

Winston, 1987.

Page 111 GAO/GGD-96-20 Federal Job Classification
Bibliography

Matthews, M.D. “Comparison of Supervisors and Incumbents, Estimates of

the Worth of Workers to their Organizations: A Brief Report.” Perceptual

and Motor Skills, Vol. 73 (1991), pp. 569-570.

Matzer, J., Jr., ed. Pay and Benefits: New Ideas for Local Government.

International City Management Association. Washington, D.C.: 1988.

McArthur, L.Z. “Social Judgment Biases in Comparable Worth Analysis,”

Comparable Worth. Washington, D.C.: National Academy Press, 1985.

McArthur, L.Z., and S.W. Obrant. “Sex Biases in Comparable Worth

Analyses.” Journal of Applied Social Psychology, Vol. 16 (1986), pp.

757-770.

McConomy, S., and B. Ganschinietz. “Trends in Job Evaluation Practices

of State Personnel Systems: 1981 Survey Findings.” Public Personnel

Management Journal, Vol. 12 (1981), pp. 1-12.

McCormick, E.J. “Job and Task Analysis,” Handbook of

Industrial/organizational Psychology, 1st ed. Chicago: Rand McNally

College Publishing Company, 1976.

McCormick, E.J. Job Analyses: Methods and Applications. New York:

Amacom, 1978.

McCormick, E.J., P.R. Jeanneret, and R.C. Mecham. Position Analysis

Questionnaire. Purdue Research Foundation, Contract No. 4497 Form B,

8-79. West Lafayette, IN: 1969.

McCormick, E.J., P.R. Jeanneret, and R.C. Mecham. “A Study of Job

Characteristics and Job Dimensions as Based on the Position Analysis

Questionnaire (PAQ).” Journal of Applied Psychology Monograph, Vol. 56

(1972), pp. 347-368.

McDonough, P., R. Snider, and J.P. Kaufman. “Male-female and

White-minority Pay Differentials in a Research Organization,”

Discrimination in Organizations: Using Social Indicators to Manage Social

Change. San Francisco: Jossey-Bass, 1979.

McDowell, D.S. “An Analysis of the National Academy of Sciences

Comparable Worth Study,” Comparable Worth: Issues and Alternatives,

2nd ed. Washington, D.C.: Equal Employment Advisory Council, 1984.

Page 112 GAO/GGD-96-20 Federal Job Classification
Bibliography

McElrath, K. “Gender, Career Disruption, and Academic Rewards.” Journal

of Higher Education, Vol. 63 (1992), pp. 269-281.

McLaughlin, S.D. “Occupational Sex Identification and the Assessment of

Male and Female Earnings in Equality.” American Sociological Review,

Vol. 43 (1978), pp. 909-921.

McMahon, W.W., and A.B. Wagner. “Expected Returns to Investment in

Higher Education.” The Journal of Human Resources, Vol. 16 (1981), pp.

274-285.

McNulty, D.J. “Differences in Pay Between Men and Women Workers.”

Monthly Labor Review, Vol. 90 (1967), pp. 40-43.

McShane, S.L. “Two Tests of Direct Gender Bias in Job Evaluation

Ratings.” Journal of Occupational Psychology, Vol. 63 (1990), pp. 129-140.

Medoff, J.L., and K.G. Abraham. “Experience, Performance, and Earnings.”

The Quarterly Journal of Economics, Vol. 95 (1980), pp. 703-736.

Medoff, J.L., and K.G. Abraham. “Are Those Paid More Really More

Productive? The Case of Experience.” The Journal of Human Resources,

Vol. 16 (1981), pp. 186-216.

Meeker, S.E. “Equal Pay, Comparable Work, and Job Evaluation.” The Yale

Law Journal, Vol. 90 (1981), pp. 657-680.

Mellon, P.M., N. Schmitt, and C. Bylenga. “Differential Predictability of

Females and Males.” Sex Roles, Vol. 6 (1980), pp. 173-177.

Meng, G.J. “All the Parts of Comparable Worth.” Personnel Journal, Vol. 69

(1990), pp. 99-104.

Messmer, D.J., and R.J. Solomon. “Differential Predictability in a Selection

Model for Graduate Students: Implications for Validity Testing.”

Educational and Psychological Measurement, Vol. 39 (1979), pp. 859-866.

Meyer, H.H. “Comparison of Foreman and General Foreman Conceptions

of the Foreman’s Job Responsibilities.” Personnel Psychology, Vol. 12

(1959), pp. 445-452.

Page 113 GAO/GGD-96-20 Federal Job Classification
Bibliography

Miceli, M.P. “Review of ’Comparable Worth: The Myth and the

Movement.’” Personnel Psychology, Vol. 38 (1985), pp. 474-478.

Michael, R.T., and H.I. Hartmann. “Pay Equity: Assessing the Issues,” Pay

Equity: Empirical Inquiries. Washington, D.C.: National Academy Press,

1989.

Miles, M.C. “Studies in Job Evaluation: Validity of a Check List for

Evaluating Office Jobs.” Journal of Applied Psychology, Vol. 36 (1952), pp.

97-101.

Milkman, R. Gender at Work: The Dynamics of the Job Segregation by Sex

During World War II. Urbana, IL: University of Illinois Press, 1987.

Milkovich, G.T. “The Emerging Debate,” Comparable Worth: Issues and

Alternatives, 2nd ed. Washington, D.C.: Equal Employment Advisory

Council, 1984.

Milkovich, G.T., and J.M. Newman. Compensation, 4th ed. Homewood, IL:

Irwin, 1993.

Milkovich, G.T., and A.K. Wigdor, eds. Pay for Performance: Evaluating

Performance Appraisal and Merit Pay. Washington, D.C.: National

Academy Press, 1991.

Millborn, S.L. A Secretary and a Cook: Challenging Women’s Wages in the

Courts of the United States and Great Britain. New York: ILR Press, 1989.

Miller, A.R., et al., eds. Work, Jobs, and Occupations: A Critical Review of

the Dictionary of Occupational Titles. Washington, D.C.: National

Academy Press, 1980.

Miller, M.M. Pay Equity in Ohio’s State Agencies: A Preliminary Report.

Ohio’s Bureau of Employment Services, Women’s Division. Columbus, OH:

1984.

Mincer, J. “The Distribution of Labor Incomes: A Survey With Special

Reference to the Human Capital Approach.” Journal of Economic

Literature, Vol. 8 (1970), pp. 1-26.

Page 114 GAO/GGD-96-20 Federal Job Classification
Bibliography

Mincer, J., and S. Polachek. “Family Investments in Human Capital:

Earnings of Women.” Journal of Political Economy, Vol. 82 (1974), pp.

S72-S108.

Miner, M.G. Job Evaluation Policies and Procedures. Bureau of National

Affairs. Washington, D.C.: 1976.

Modernizing Federal Classification: An Opportunity for Excellence.

National Academy of Public Administration. Washington, D.C.: 1991.

Montgomery, E., and W. Wascher. “Race and Gender Wage Inequality in

Services and Manufacturing.” Industrial Relations, Vol. 26 (1987), pp.

284-290.

Moore, W.J., D.K. Pearce, and R.M. Wilson. “The Regulation of

Occupations and the Earnings of Women.” The Journal of Human

Resources, Vol. 16 (1981), pp. 366-383.

Moore, W.J. and J. Raisian. “Government Wage Differentials Revisited.”

Journal of Labor Research, Vol. 12 (1991), 13-33.

Moore, L.M., and A.U. Rickel. “Characteristics of Women in Traditional

and Non-traditional Managerial Roles.” Personnel Psychology, Vol. 33

(1980), pp. 317-333.

Moroney, J.R., ed. Income Inequality: Trends and International 

Comparisons. Lexington, MA: Lexington Books, 1979.

Morse, P.K. “Detection of Sex-related Salary Discrimination: A

Demonstration Using Constructed Data,” Salary Equity: Detecting Sex Bias

in Salaries Among College and University Professors. Lexington, MA:

Lexington Books, 1979.

Motowidlo, S.J. “Relationship Between Self-rated Performance and Pay

Satisfaction Among Sales Representatives.” Journal of Applied

Psychology, Vol. 67 (1982), pp. 209-213.

Mount, M.K., and R.A. Ellis. “Investigation of Bias in Job Evaluation

Ratings of Comparable Worth Study Participants.” Personnel Psychology,

Vol. 40 (1987), pp. 85-96.

Page 115 GAO/GGD-96-20 Federal Job Classification
Bibliography

Mount, M.K., and R.A. Ellis. “Sources of Bias in Job Evaluation: A Review

and Critique of Research.” Journal of Social Issues, Vol. 45 (1989), pp.

153-167.

Moyer, K.L. Sex Differences in Vocational Aspirations and Expectations of

Pennsylvania 11th Grade Students. Pennsylvania Department of

Education, 1978.

Muffo, J.A., L. Braskamp, and I.W. Langston, IV. “Equal Pay for Equal

Qualifications? A Model for Determining Race or Sex Discrimination in

Salaries,” Salary Equity: Detecting Sex Bias in Salaries Among College and

University Professors. Lexington, MA: Lexington Books, 1979.

Mulcahy, R.W., and J.E. Anderson. “The Bargaining Battleground Called

Comparable Worth.” Public Personnel Management, Vol. 15 (1986), pp.

233-247.

Mullins, W.C., and W.W. Kimbrough. “Group Composition as a

Determinant of Job Analysis Outcomes.” Journal of Applied Psychology,

Vol. 73 (1988), pp. 657-664.

Mumford, M.D., et al. “Measuring Occupational Difficulty: A Construct

Validation Against Training Criteria.” Journal of Applied Psychology, Vol.

72 (1987), pp. 578-587.

Murphy, K.R. “Difficulties in a Statistical Control of Halo.” Journal of

Applied Psychology, Vol. 67 (1982), pp. 161-164.

Muson, H. “Hard-hat Women.” Across the Board, Vol. 18 (1981), pp. 12-18.

Myers, J.H. “An Experimental Investigation of “Point” Job Evaluation.”

Journal of Applied Psychology, Vol. 42 (1958), pp. 357-361.

Myers, J.H. “Removing Halo from Job Evaluation Factor Structure.”

Journal of Applied Psychology, Vol. 49 (1965), pp. 217-221.

Nash, A.N., and S.J. Carroll. The Management of Compensation. Monterey,

CA: Brooks/Cole Publishing Company, 1975.

National Survey of Professional, Administrative, Technical, and Clerical

Pay. U.S. Department of Labor, Bulletin 2271. Washington, D.C.: 1986.

Page 116 GAO/GGD-96-20 Federal Job Classification
Bibliography

Naughton, T.J. “Effect of Female-linked Job Titles on Job Evaluation

Ratings.” Journal of Management, Vol. 14 (1988), pp. 567-578.

Neigenfind, J.L. “Position Accuracy Certification.” Public Personnel

Management Journal, Vol. 11 (1982), pp. 213-218.

Nelson, B.A., E.M. Opton, and T.E. Wilson. “Wage Discrimination and the

“Comparable Worth” Theory in Perspective.” University of Michigan

Journal of Law Reform, Vol. 13 (1980), pp. 1-301.

Newman, W. “Pay Equity Emerges as a Top Labor Issue in the 1980s.”

Monthly Labor Review, Vol. 105 (1982), pp. 49-51.

Newman, W. “Statement to the Equal Pay Joint Committee.” Public

Personnel Management Journal, Vol. 12 (1983), pp. 382-389.

Niemi, A.W. “Discrimination Against Women Reconsidered.” American

Journal of Economics and Sociology, Vol. 38 (1979), pp. 291-292.

Nieva, V.F., and B.A. Gutek. “Sex Effects on Evaluation.” Academy of

Management Review, Vol. 5 (1980), pp. 267-276.

Note. “Equal Pay, Comparable Work, and Job Evaluation.” The Yale Law

Journal, Vol. 90 (1981), pp. 657-680.

Odel, D. “A Failed Experiment.” The National Law Journal, Vol. 14 (1991),

pp. 13-14.

Oldham, G.R., and H.E. Miller. “The Effect of Significant Others Job

Complexity on Employee Reactions to Work.” Human Relations, Vol. 32

(1979), pp. 247-260.

Olmstead, A.L., and S.M. Sheffrin. “The Medical School Admission Process:

An Empirical Investigation.” The Journal of Human Resources, Vol. 16

(1981), pp. 459-467.

Olney, P.B., Jr. “Meeting the Challenge of Comparable Worth: Part 1.”

Compensation and Benefits Review, Vol. 19 (1987), pp. 34-44.

Olson, C.A. “An Analysis of Wage Differentials Received by Workers on

Dangerous Jobs.” Journal of Human Resources, Vol. 16 (1981), pp. 167-185.

Page 117 GAO/GGD-96-20 Federal Job Classification
Bibliography

O’Neill, J. “Role Differentiation and the Gender Gap in Wage Rates,”

Women and Work 1: An Annual Review. Beverly Hills, CA: Sage

Publications, Inc., 1985.

O’Neill, J., M. Brien, and J. Cunningham. “Effects of Comparable Worth

Policy: Evidence from Washington State.” American Economic Review,

Vol. 79 (1989), pp. 305-309.

Oppenheimer, V. “The Sex Labeling of Jobs.” Industrial Relations, Vol. 7

(1968), pp. 219-234.

Orazem, P.F., and J.P. Mattila. Comparable Worth and the Structure of

Earnings: The Iowa Case. Iowa State University, 1987.

Orazem, P.F., J.P. Mattila, and C.Y. Ruoh. “An Index Number Approach to

the Measurement of Wage Differentials by Sex.” Journal of Human

Resources, Vol. 25 (1990), pp. 125-136.

Orazem, P.F., J.P. Mattila, and S.K. Weikum. “Comparable Worth Plans and

Factor Point Pay Analysis in State Government.” Industrial Relations, Vol.

31 (1992), pp. 195-215.

O’Reilly, A.P. “Skill Requirements: Supervisor-subordinate Conflict.”

Personnel Psychology, Vol. 26 (1973), pp. 75-80.

Otis, J.L., and R.H. Leukart. Job Evaluation: A Basis for Sound Wage

Administration. Englewood Cliffs, NJ: Prentice Hall, 1954.

Ott, M.D. “Retention of Men and Women Engineering Students.” Research

in Higher Education, Vol. 9 (1978), pp. 137-150.

Over, R. “Research Impact of Men and Women Social Psychologists.”

Personality and Social Psychology Bulletin, Vol. 7 (1981), pp. 596-599.

Paglin, M., and A.M. Rufolo. “Heterogeneous Human Capital, Occupational

Choice, and Male-female Earnings Differences.” Journal of Labor

Economics, Vol. 8 (1990), pp. 123-144.

Paisner, A.M. “BLS Regional Offices: Contribution to Wage Programs.”

Monthly Labor Review, Vol. 115 (1992), pp. 30-34.

Page 118 GAO/GGD-96-20 Federal Job Classification
Bibliography

Palmer, P., and R.M. Spalter-Roth. Gender Practices and Employment: The

Sears Case and the Issue of “Choice”. Institute of Women’s Policy

Research. Washington, D.C.

Paterson, T.T. Job Evaluation Volume 1—A New Method. London:

Business Books Unlimited, 1972.

Paterson, T.T. Job Evaluation Volume 2—A Manual for the Paterson

Method. London: Business Books Unlimited, 1972.

Paterson, T.T., and T.M. Husband. “Decision-making Responsibility:

Yardstick for Job Evaluation.” Compensation Review, Vol. 2 (1970), pp.

21-31.

Patten, T.H. Fair Pay: The Managerial Challenge of Comparable Job Worth

and Job Evaluation. San Francisco: Jossey-Bass, 1988.

Patton, J.A., C.L. Littlefield, and S.A. Self. Job Evaluation: Text and Cases,

3rd ed. Homewood, IL: Richard D. Irwin, 1964.

Paul, E.F. Equity and Gender: The Comparable Worth Debate. New

Brunswick, NJ: Transaction Publishers, 1989.

Pay Equity in Ohio’s State Agencies. State of Ohio, Ohio’s Bureau of

Employment Services, Women’s Division. Columbus, OH: 1986.

Pay Equity: The Minnesota Experience. Commission on the Economic

State of Women, 1989.

Penner, M. “How Job-based Classification Systems Promote

Organizational Ineffectiveness.” Public Personnel Management, Vol. 12

(1983), pp. 268-276.

Perrin, S.M. Comparable Worth and Public Policy: The Case of

Pennsylvania. Philadelphia: Industrial Research Unit, The Wharton School,

University of Pennsylvania, 1985.

Perrucci, C.C., and D.B. Targ. “Early Work Orientation and Later

Situational Factors as Elements of Work Commitment Among Married

Women College Graduates.” The Sociological Quarterly, Vol. 19 (1978), pp.

266-280.

Page 119 GAO/GGD-96-20 Federal Job Classification
Bibliography

Personnel Research Bibliography on Job Evaluation. U.S. Office of

Personnel Management. Washington, D.C.: 1990.

Perspectives on Working Women: A Data Book. U.S. Department of Labor,

Bureau of Labor Statistics, Bulletin 2080. Washington, D.C.: 1980.

Peters, L.H., et al. “Sex Bias and Managerial Evaluations: A Replication and

Extension.” Journal of Applied Psychology, Vol. 69 (1984), pp. 349-352.

Peterson, J. “The Challenge of Comparable Worth: An Institutionalist

View.” Journal of Economic Issues, Vol. 24 (1990), pp. 605-612.

Pezzullo, T.R., and B.E. Brittingham, ed. Salary Equity: Detecting Sex Bias

in Salaries Among College and University Professors. Lexington, MA:

Lexington Books, 1979.

Pezzullo, T.R., and B.E. Brittingham. “The Assessment of Salary Equity: A

Methodology, Alternatives, and a Dilemma,” Salary Equity: Detecting Sex

Bias in Salaries Among College and University Professors. Lexington, MA:

Lexington Books, 1979.

Pfeffer, J., and J. Ross. “Union-nonunion Effects on Wage and Status

Attainment.” Industrial Relations, Vol. 19 (1980), pp. 140-151.

Pfeffer, J., and J. Ross. “The Effects of Marriage and a Working Wife on

Occupational and Wage Attainment.” Administrative Science Quarterly,

Vol. 27 (1982), pp. 66-80.

Pfeffer, J., and J. Ross. “Gender-based Wage Differences: The Effects of

Organizational Context.” Work and Occupations, Vol. 17 (1990), pp. 55-78.

Phillips, A., and B. Taylor. “Sex and Skill: Notes Towards a Feminist

Economics.” Feminist Review, Vol. 6 (1980), pp. 79-88.

Phillips, M.D., and R.L. Pepper. “Shipboard Fire-fighting Performance of

Females and Males.” Human Factors, Vol. 24 (1982), pp. 277-283.

Piliavin, J.A., and R.K. Unger. “The Helpful but Helpless Female: Myth or

Reality,” Women, Gender, and Social Psychology. Hillsdale, NJ: Lawrence

Erlbaum Associates, 1985.

Page 120 GAO/GGD-96-20 Federal Job Classification
Bibliography

Polachek, S.W. “Potential Biases in Measuring Male-female

Discrimination.” Journal of Human Resources, Vol. 10 (1975), pp. 205-229.

Polachek, S.W. “Sex Differences in College Major.” Industrial and Labor

Relations Review, Vol. 31 (1978), pp. 498-508.

Polachek, S.W. “Occupational Self-selection: A Human Capital Approach

to Sex Differences in Occupational Structure.” Review of Economics and

Statistics, Vol. 63 (1981), pp. 60-69.

Pommerenke, P.L. “Comparable Worth: A Panacea for Discrimination

Against Women in the Labor Market?” American Economist, Vol. 32

(1988), pp. 44-48.

Powers, T.N. “An Idea With a Long Way to Go.” American Bar Association

Journal, Vol. 70 (1984), pp. 16-21.

Pratt, L.J. “Local Government and Comparable Worth: A Case Study.”

Document, City of Chattanooga Study. Chattanooga, TN: 1988.

Pratt, L.J., S.A. Smullen, and B.L. Kyer. “The Macroeconomics of the Equal

Pay Act.” Journal of Macroeconomics, Vol. 12 (1990), pp. 675-689.

Prediger, D.J. “The Determination of Holland Types Characterizing

Occupational Groups.” Journal of Vocational Behavior, Vol. 16 (1980), pp.

33-42.

Prien, E.P., G.V. Barrett, and B. Svetlik. “Use of Questionnaires in Job

Evaluation.” Journal of Industrial Psychology, Vol. 3 (1965), pp. 91-94.

Prien, E.P., I.L. Goldstein, and W.H. Macey. “Multidomain Job Analysis:

Procedures and Application.” Training and Development Journal, Vol. 41

(1987), pp. 68-72.

Prien, E.P., and S.D. Saleh. “A Study of Bias in Job Analysis.” Journal of

Industrial Psychology, Vol. 22, pp. 113-117.

Primack, R.B., and V.E. O’Leary. “Research Productivity of Men and

Women Ecologists: A Longitudinal Study of Former Graduate Students.”

The Ecological Society of America Bulletin, Vol. 70 (1989), pp. 7-21.

Page 121 GAO/GGD-96-20 Federal Job Classification
Bibliography

Ragan, J.F., and S.P. Smith. “The Impact of Differences in Turnover Rates

on Male-female Pay Differentials.” Journal of Human Resources, Vol. 16

(1981), pp. 343-365.

Ramsey, G.A. “A Generalized Multiple Regression Model for Predicting

College Faculty Salaries and Estimating Sex Bias,” Salary Equity:

Detecting Sex Bias in Salaries Among College and University Professors.

Lexington, MA: Lexington Books, 1979.

Ratner, R.S., ed. Equal Employment Policy for Women: Strategies for

Implementation in United States, Canada, Western Europe. Philadelphia:

Temple University Press, 1980.

Ray, J.M., and A.B. Rubin. “Pay Equity for Women in Academic Libraries:

An Analysis of ARL Salary Surveys, 1976/77-1983/84.” College and

Research Libraries, Vol. 48 (1987), pp. 36-49.

Raymond, R.D., M.L. Sesnowitz, and D.R. Williams. “Does Sex Still Matter?

New Evidence From the 1980s.” Economic Inquiry, Vol. 26 (1988), pp.

43-58.

Rea, L.M., and R.A. Parker. Designing and Conducting Survey Research: A

Comprehensive Guide. San Francisco: Jossey-Bass Publishers, 1992.

Reichenberg, N.E. “Pay Equity in Review.” Public Personnel Management,

Vol. 15 (1986), pp. 211-231.

Reis, H.T., and L.A. Jackson. “Sex Differences in Reward Allocation:

Subjects, Partners, and Tasks.” Journal of Personality and Social

Psychology, Vol. 40 (1981), pp. 465-478.

Remick, H. “The Comparable Worth Controversy.” Public Personnel

Management, Vol. 10 (1981), pp. 371-383.

Remick, H. “An Update on Washington State.” Public Personnel

Management Journal, Vol. 12 (1983), pp. 390-394.

Remick, H. “Dilemmas of Implementation: The Case of Nursing,”

Comparable Worth and Wage Discrimination: Technical Possibilities and

Political Realities. Philadelphia: Temple University Press, 1984.

Page 122 GAO/GGD-96-20 Federal Job Classification
Bibliography

Remick, H. “Major Issues in A Priori Applications,” Comparable Worth and

Wage Discrimination: Technical Possibilities and Political Realities.

Philadelphia: Temple University Press, 1984.

Remick, H., ed. Comparable Worth and Wage Discrimination: Technical

Possibilities and Political Realities. Philadelphia: Temple University Press,

1984.

Remick, H. “The Case of Comparable Worth in Washington State.” Policy

Studies Review, Vol. 5 (1986), pp. 838-848.

Remick, H., and R.J. Steinberg. “Technical Possibilities and Political

Realities: Concluding Remarks,” Comparable Worth and Wage

Discrimination: Technical Possibilities and Political Realities.

Philadelphia: Temple University Press, 1984.

Report of Wisconsin’s Task Force on Comparable Worth. University of

Wisconsin-Madison, Wisconsin Task Force on Comparable Worth.

Madison, WI: 1986.

Research on Evaluator Bias. Organizational Research and Development,

Inc. Columbus, OH.

Reskin, B.F., ed. Sex Segregation in the Workplace. Washington, D.C.:

National Academy Press, 1984.

Reskin, B.F., and H.I. Hartmann, eds. Women’s Work, Men’s Work: Sex

Segregation on the Job. Washington, D.C.: National Academy Press, 1986.

Reskin, B., et al. “Salary-setting Practices That Unfairly Disadvantage

Women.” Academe, Vol. 78 (1992), pp. 32-35.

Reskin, B.F., and P.A. Roos. Job Queues, Gender Queues: Explaining

Women’s Inroads Into Male Occupations. Philadelphia: Temple University

Press, 1990.

Rhoads, S.E. Incomparable Worth: Pay Equity Meets the Market.

Cambridge: Cambridge University Press, 1993.

Rhynes, S.L., D.P. Schwab, and H.G. Heneman, III. “The Role of Pay and

Market Pay Variability in Job Application Decisions.” Organizational

Behavior and Human Performance, Vol. 31 (1983), pp. 353-364.

Page 123 GAO/GGD-96-20 Federal Job Classification
Bibliography

Rice, R.W., D. Instone, and J. Adams. “Leader Sex, Leader Success, and

Leadership Process: Two Field Studies.” Journal of Applied Psychology,

Vol. 69 (1984), pp. 12-31.

Risher, H. “Job Evaluation: Problems and Prospects.” Personnel, Vol. 61

(1984), pp. 53-66.

Risher, H. and C. Fay. “Federal Pay Reform: A Response to an Emerging

Crisis.” Public Personnel Management, Vol. 20 (1991), pp. 385-395.

Ritchie, R.J., and V.D. Beardsley. “A Market Research Approach to

Determine Local Labor Market Availability for Non-management Jobs.”

Personnel Psychology, Vol. 31 (1978), pp. 449-459.

Rizzo, A., and C. Mendez. The Integration of Women in Management: A

Guide for Human Resources and Management Development Specialists.

New York: Quorum Books, 1990.

Roback, J. A Matter of Choice: A Critique of Comparable Worth by a

Skeptical Feminist. New York: Priority Press Publication, 1986.

Robertson, T.M. “Fundamental Strategies for Wage and Salary

Administration.” Personnel Journal, Vol. 65 (1986), pp. 120-132.

Robinson, D.D., O.W. Wahlstrom, and R.C. Mecham. “Comparison of Job

Evaluation Methods: A ‘Policy-capturing’ Approach Using the Position

Analysis Questionnaire.” Journal of Applied Psychology, Vol. 59 (1974), pp.

633-637.

Robinson, M.D., and P.V. Wunnava. “Measuring Direct Discrimination in

Labor Markets Using a Frontier Approach: Evidence from CPS Female

Earnings Data.” Southern Economic Journal, Vol. 56 (1989), pp. 212-218.

Rock, M.L., ed. Handbook of Wage and Salary Administration, 2nd ed. New

York: McGraw-Hill, 1984.

Rock, M.L., and L.A. Berger, eds. A Compensation Handbook. New York:

McGraw-Hill, 1991.

Rogers, R.C. “Analysis of Two Point-rating Job Evaluation Plans.”

Personnel Psychology, Vol. 30 (1946), pp. 579-585.

Page 124 GAO/GGD-96-20 Federal Job Classification
Bibliography

Roos, P.A. “Sex Stratification in the Workplace: Male-female Difference in

Economic Returns to Occupation.” Social Science Research, Vol. 10

(1981), pp. 195-223.

Roose, J.E., and M.E. Doherty. “A Social Judgement Theoretic Approach to

Sex Discrimination in Faculty Salaries.” Organizational Behavior and

Human Performance, Vol. 22 (1978), pp. 193-215.

Root, N. and J.R. Daley. “Are Women Safe Workers? A New Look at the

Data.” Monthly Labor Review, Vol. 103 (1980), pp. 3-10.

Rosenbaum, J.E. “Organizational Career Mobility: Promotion Chances in a

Corporation During Periods of Growth and Contraction.” American

Journal of Sociology, Vol. 85 (1979), pp. 21-48.

Rosenbaum, J.E. “Tournament Mobility: Career Patterns in a Corporation.”

Administrative Science Quarterly, Vol. 24 (1979), pp. 220-241.

Rosenbaum, J.E. “Hierarchical and Individual Effects on Earnings.”

Industrial Relations, Vol. 19 (1980), pp. 1-14.

Rosengren, K.E., ed. Advances in Content Analysis. Beverly Hills, CA:

Sage, 1981.

Rotter, N.G. “Perceived Commitment, Salary Recommendation, and

Employees’ Sex.” Perceptual and Motor Skills, Vol. 64 (1987), pp. 651-658.

Rousseau, D.M. “Job Perceptions When Working With Data, People, and

Things.” Journal of Occupational Psychology, Vol. 55 (1982), pp. 43-52.

Ruderfer, E.D. “Sex-based Wage Discrimination Under Title VII: Equal Pay

for Equal Work or Equal Pay for Comparable Work?” William and Mary

Law Review, Vol. 22 (1981), pp. 421-485.

Rumberger, R.W. “The Changing Skill Requirements of Jobs in the U.S.

Economy.” Industrial and Labor Relations Review, Vol. 34 (1981), pp.

578-590.

Rush, C.H., and R.M. Bellows. “Job Evaluation for a Small Business.”

Personnel Psychology, Vol. 2 (1949), pp. 301-310.

Page 125 GAO/GGD-96-20 Federal Job Classification
Bibliography

Rynes, S.L., and G.T. Milkovich. “Wage Surveys: Dispelling Some Myths

About the ‘Market Wage.’” Personnel Psychology, Vol. 39 (1986), pp. 71-90.

Rynes, S.L., C.L. Weber, and G.T. Milkovich. “Effects of Market Survey

Rates, Job Evaluation, and Job Gender on Job Pay.” Journal of Applied

Psychology, Vol. 74 (1989), pp. 114-123.

Rytina, N.F. “Occupational Segregation and Earnings Differences by Sex.”

Monthly Labor Review, Vol. 104 (1981), pp. 49-53.

Rytina, N.F. “Earnings of Men and Women: A Look at Specific

Occupations.” Monthly Labor Review, Vol. 105 (1982), pp. 25-31.

Rytina, N.F., and S.M. Bianchi. “Occupational Reclassification and

Changes in Distribution by Gender.” Monthly Labor Review, Vol. 107

(1984), pp. 11-17.

Sackett, P.R., E.T. Cornelius, and T.J. Carron. “A Comparison of Global

Judgement vs. Task-oriented Approaches to Job Classification.” Personnel

Psychology, Vol. 34 (1981), pp. 791-804.

Sahl, R.J. “How to Install a Job-evaluation.” Personnel, Vol. 66 (1989), pp.

38-42.

Sanborn, H. “Paid Differences Between Men and Women.” Industrial and

Labor Relations Review, Vol. 17 (1964), pp. 534-550.

Sanchez, J.I., and S.L. Fraser. “On the Choice of Scales for Task Analysis.”

Journal of Applied Psychology, Vol. 77 (1992), pp. 545-553.

Sanchez, J.I., and E.L. Levine. “Determining Important Tasks Within Jobs:

A Policy Capturing Approach.” Journal of Applied Psychology, Vol. 78

(1989), pp. 336-342.

Sandy, P.R. Female Power and Male Dominance: On the Origins of Sexual

Inequality. New York: Cambridge University Press, 1981.

Sape, G.P. “Coping with Comparable Worth.” Harvard Business Review,

Vol. 63 (1985), pp. 145-152.

Page 126 GAO/GGD-96-20 Federal Job Classification
Bibliography

Satter, G.A. “Method of Paired Comparisons and a Specification Scoring

Key in the Evaluation of Jobs.” Journal of Applied Psychology, Vol. 33

(1949), pp. 212-221.

Sauer, R.L. “Measuring Relative Worth of Managerial Positions.”

Compensation Review, Vol. 4, pp. 9-18.

Sayles, L.R. “Worker Values in Job Evaluation: Impact of Job Evaluation

on Worker Attitudes.” Personnel, Vol. 30 (1954), pp. 266-274.

Scanlan, J.P. “Illusions of Job Segregation.” Public Interest, Vol. 39 (1988),

pp. 54-69.

Schein, V.E. “Relationships Between Sex Role Stereotypes and Requisite

Management Characteristics Among Female Managers.” Journal of Applied

Psychology, Vol. 60 (1975), pp. 340-344.

Schleifer, L.M., and O.G. Okogbaa. “System Response Time and Method of

Pay: Cardiovascular Stress Effects in Computer-based Tasks.”

Ergonomics, Vol. 33 (1990), pp. 1495-1509.

Schmitt, N., B.W. Coyle, and P.M. Mellon. “Subgroup Differences in

Predictor and Criterion Variances and Differential Validity.” Journal of

Applied Psychology, Vol. 63 (1978), pp. 667-672.

Scholl, R.W., and E.A. Cooper. “The Use of Job Evaluation to Eliminate

Gender Based Pay Differentials.” Public Personnel Management, Vol. 20

(1991), pp. 1-18.

Schonberger, R., and H. Hennessey, Jr. “Is Equal Pay for Comparable Work

Fair?” Personnel Journal, Vol. 60 (1981), pp. 964-968.

Schroeder, P., and C. Horner. “Comparable Worth: A Wrong Turn.” The

Bureaucrat, Vol. 16 (1987/88), pp. 4-9.

Schuster, J. “How to Control Job Evaluation Inflation.” Personnel

Administrator, Vol. 30, pp. 167-172.

Schuster, M. “The Scanlon Plan: A Longitudinal Analysis.” Journal of

Applied Behavioral Science, Vol. 20 (1984), pp. 23-38.

Page 127 GAO/GGD-96-20 Federal Job Classification
Bibliography

Schwab, D.P. “Job Evaluation and Pay Setting: Concepts and Practices,”

Comparable Worth: Issues and Alternatives, 2nd ed. Washington, D.C.:

Equal Employment Advisory Council, 1984.

Schwab, D.P. “Job Evaluation Research and Research Needs,” Comparable

Worth: New Directions for Research. Washington, D.C.: National Academy

Press, 1985.

Schwab, D.P. “Contextual Variables in Employee Performance-turnover

Relationships.” Academy of Management Journal, Vol. 34 (1991), pp.

966-975.

Schwab, D.P., and R. Grams. “Sex-related Errors in Job Evaluation: A

‘Real-world’ Test.” Journal of Applied Psychology, Vol. 70 (1985), pp.

533-539.

Schwab, D.P., and H.G. Heneman. “Assessment of a Consensus-based

Multiple Information Source Job Evaluation System.” Journal of Applied

Psychology, Vol. 71 (1986), pp. 354-356.

Schwab, D., and C. Olson. “Merit Pay Practices,” Do Compensation

Practices Matter? Ithaca, NY: ILR Press, 1990.

Schwab, D.P., and D.W. Wichern. “Systematic Bias in Job Evaluation and

Market Wages: Implications for the Comparable Worth Debate.” Journal of

Applied Psychology, Vol. 68 (1983), pp. 60-69.

Scott, J.A., and R. Zickefoose. “Achieving Comparable Worth in a

Non-union Municipal Government Setting: The Chemistry’s Right in

Colorado Springs.” Presented at the American Society for Public

Administration Conference, 1983.

Scozzaro, P.P., L.M. Subich. “Gender and Occupational Sex-type

Differences in Job Outcome Factor Perceptions.” Journal of Vocational

Behavior, Vol. 36 (1990), pp. 109-119.

Seaman, F., and A. Lorimer. Winning at Work: A Book for Women.

Philadelphia: Running Press, 1979.

Seashore, H.G. “Women are More Predictable Than Men.” Journal of

Counseling Psychology, Vol. 9 (1962), pp. 261-270.

Page 128 GAO/GGD-96-20 Federal Job Classification
Bibliography

Shaffer, L.J., and R.M. Wilson. “Racial Discrimination in Occupational

Choice.” Industrial Relations, Vol. 19 (1980), pp. 199-205.

Shaw, K.L. “The Income Effects of Occupational Change and the

Investment in Occupational Skills.” Ph.D. Thesis, Harvard University, 1981.

Shimmin, S. “Job Evaluation and Equal Pay for Work of Equal Value.”

Applied Psychology—An International Review, Vol. 36 (1987), pp. 61-70.

Sibson, R.E. Compensation, 5th ed. New York: American Management

Association, 1990.

Sigelman, L., H.B. Milward, and J.M. Shepard. “The Salary Differential

Between Male and Female Administrators: Equal Pay for Equal Work?”

Academy of Management Journal, Vol. 25 (1982), pp. 664-671.

Sinnott, P.A. “The Comparable Worth Controversy.” Journal of Career

Planning and Employment, Vol. 45 (1985), pp. 46-51.

Smith, B.N., et al. “What Is In a Name: The Impact of Job Titles on Job

Evaluation Results.” Journal of Business and Psychology, Vol. 3 (1989), pp.

341-351.

Smith, B.N., P.G. Benson, and J.S. Hornsby. “The Effects of Job

Description Content on Job Evaluation Judgements.” Journal of Applied

Psychology, Vol. 75 (1990), pp. 301-309.

Smith, F.S. “Compensating Wage Differentials and Public Policy: A

Review.” Industrial and Labor Relations Review, Vol. 32 (1979), pp.

339-352.

Smith, J. “Comparable Worth, Gender, and Human Capital Theory,”

Ingredients for Women’s Employment Policy. Albany: State University of

New York Press, 1987.

Smith, J.E., and M.D. Hakel. “Convergence Among Data Sources, Response

Bias, and Reliability and Validity of a Structured Job Analysis

Questionnaire.” Personnel Psychology, Vol. 32 (1979), pp. 677-691.

Smyth, R.C. “How to Rank and Price Management Jobs.” Factory

Management and Maintenance, Vol. 108 (1950), pp. 116-117.

Page 129 GAO/GGD-96-20 Federal Job Classification
Bibliography

Snelgar, R.J. “The Comparability of Job Evaluation Methods in Supplying

Approximately Similar Classifications in Rating One Job Series.” Personnel

Psychology, Vol. 36 (1983), pp. 371-380.

Solomon, R.J. “Determining the Fairness of Salary in Public Employment.”

Public Personnel Management Journal, Vol. 8 (1980), pp. 154-159.

Sorenson, E. (1989). “Measuring the Pay Disparity Between Typically

Female Occupations and Other Jobs: A Bivariate Selectivity Approach.”

Industrial and Labor Relations Review, Vol. 42, pp. 624-639.

Spalter-Roth, R., and H. Hartmann. Raises and Recognition: Secretaries,

Clerical Workers and the Union Wage Premium. Institute for Women’s

Policy Research. Washington, D.C.: 1990.

Spang, S.D. “Pay Equity by Committee: A Labor-management Case Study.”

Cupa Journal, Vol. 41 (1990), pp. 21-34.

Spangler, E., M.A. Gordon, and R.M. Pipkin. “Token Women: An Empirical

Test of Kantor’s Hypothesis.” American Journal of Sociology, Vol. 84

(1978), pp. 160-170.

Sparks, C.P. “Job Analysis,” Personnel Management. Boston: Allyn and

Bacon, 1982.

Spelfogel, E.J. “Equal Pay for Work of Comparable Value: A New

Concept.” Labor Law Review, Vol. 32 (1981), pp. 30-39.

Spence, J.T. and L.L. Sawin. “Images of Masculinity and Femininity: A

Reconceptualization,” Women, Gender, and Social Psychology. Hillsdale,

NJ: Lawrence Erlbaum Associates, 1985.

Spriegel, W.R., and E. Lanham. “Job Evaluation in Department Stores.”

Journal of Retailing, Vol. 27 (1951), pp. 79-85.

Stanley, J.C., and A.C. Porter. “Correlation of Scholastic Aptitude Test

Score With College Grades for Negroes Versus Whites.” Journal of

Educational Measurement, Vol. 4 (1967), pp. 199-218.

Steel, B.S., and N.P. Lovrich. “Comparable Worth: The Problematic

Politicization of a Public Personnel Issue.” Public Personnel Management,

Vol. 16 (1987), pp. 23-36.

Page 130 GAO/GGD-96-20 Federal Job Classification
Bibliography

Steiger, Fink, and Kosecoff, Inc. Literature and Secondary Data Review of

the Vocational Education Equity Study—Final Report. American Institutes

for Research. Palo Alto, CA: 1979.

Steinberg, R. “‘A Want of Harmony’: Perspectives on Wage Discrimination

and Comparable Worth,” Comparable Worth and Wage Discrimination:

Technical Possibilities and Political Realities. Philadelphia: Temple

University Press, 1984.

Steinberg, R.J. “Identifying Wage Discrimination and Implementing Pay

Equity Adjustments,” Comparable Worth: Issues for the 80s. U.S.

Commission on Civil Rights, ed. Washington, D.C.: 1984.

Steinberg, R.J. “Evaluating Jobs.” Society, Vol. 22 (1985), pp. 44-54.

Steinberg, R. “The Debate on Comparable Worth.” New Politics, Vol. 1

(1986), pp. 108-126.

Steinberg, R. “Radical Challenges in a Liberal World: The Mixed Success of

Comparable Worth.” Gender and Society, Vol. 1 (1987), pp. 466-475.

Steinberg, R.J. “Social Construction of Skill: Gender, Power, and

Comparable Worth.” Work and Occupations, Vol. 17 (1990), pp. 449-482.

Steinberg, R., and L. Haignere. “Equitable Compensation: Methodological

Criteria for Comparable Worth,” Ingredients for Women’s Employment

Policy. Albany, NY: State University of New York Press, 1987.

Steinberg, R., et al. The New York Pay Equity Study: A Research Report.

Center for Women in Government, State University of New York at Albany,

Albany, NY: 1986.

Steinberg, R., and S. Shapiro. “Sex Differences in Personality Traits of

Female and Male Master of Business Administration Students.” Journal of

Applied Psychology, Vol. 67 (1982), pp. 306-310.

Stewart, D.A. “Improving Job Evaluation Results.” Personnel, Vol. 25

(1949), pp. 356-365.

Stigers, M.F., and E.G. Reed. The Theory and Practice of Job Rating. New

York: McGraw-Hill, 1942.

Page 131 GAO/GGD-96-20 Federal Job Classification
Bibliography

Stockard, J., et al. Sex Equity in Education. New York: Academic Press,

1980.

Stokey, N.L. “Job Differentiation and Wages.” The Quarterly Journal of

Economics, Vol. 95 (1980), pp. 431-449.

Stroh, L.K., J.M. Brett, and A.H. Reilly. “All the Right Stuff: A Comparison

of Female and Male Managers’ Career Progression.” Journal of Applied

Psychology, Vol. 77 (1992), pp. 251-260.

Study on Setting Salaries in the Classified Service. State of Nevada,

Department of Personnel. Carson City, NV: 1985.

Stutz, R.L. and H.E. Smalley. “Management, Union Join in Job Evaluation.”

Personnel Journal, Vol. 34 (1950), pp. 412-416.

Subich, L.M., et al. “Occupational Perceptions of Males and Females as a

Function of Sex Ratios, Salary, and Availability.” Journal of Vocational

Behavior, Vol. 28 (1986), pp. 123-134.

Suits, D.B. “Dummy Variables: Mechanics v. Interpretation.” Review of

Economics and Statistics, Vol. 66 (1984), pp. 177-180.

Sullivan, J.F. “Comparable Worth and the Statistical Audit of Pay Programs

for Illegal Systemic Discrimination.” Personnel Administrator, Vol. 30

(1985), pp. 102-111.

Summers, T.P. “Examination of Sex Differences in Expectations of Pay

and Perceptions of Equity in Pay.” Psychological Reports, Vol. 62 (1988),

pp. 491-496.

Summers, T.P., and W.H. Hendrix. “Modeling the Role of Pay Equity

Perceptions: A Field Study.” Journal of Occupational Psychology, Vol. 64

(1991), pp. 145-157.

Svetlik, B., E. Prien, and G. Barrett. “Relationships Between Job Difficulty,

Employee’s Attitude Toward His Job, and Supervisory Ratings of the

Employee Effectiveness.” Journal of Applied Psychology, Vol. 48 (1964),

pp. 320-324.

Page 132 GAO/GGD-96-20 Federal Job Classification
Bibliography

Szmania, J.M., and D. Doverspike. “A Test of the Differential Investment

Hypothesis Applied to the Exploration of Gender Differences in Reactions

to Pay.” Journal of Vocational Behavior, Vol. 37 (1990), pp. 239-250.

Szwajklowski, E., and L. Larwood. “Rational Decision Processes and Sex

Discrimination: Testing ‘Rational’ Bias Theory.” Journal of Organizational

Behavior, Vol. 12 (1991), pp. 507-527.

Taber, T.D., T.A. Beehr, and J.T. Walsh. “Relationships Between Job

Evaluation Ratings and Self-ratings of Job Characteristics.” Organizational

Behavior and Human Decision Processes, Vol. 35 (1985), pp. 27-45.

Taber, T.D., and T.D. Peters. “Assessing the Completeness of a Job

Analysis Procedure.” Journal of Organizational Behavior, Vol. 12 (1991),

pp. 581-593.

Taeuber, C. Statistical Handbook on Women in America. Phoenix, AZ:

Oryx Press, 1991.

Task Force Findings, Conclusions and Recommendations. Employees Pay

Equity/Comparable Worth Task Force. Davis, CA: 1985.

Taubman, P.J., and T.J. Wales. “Higher Education, Mental Ability, and

Screening.” Journal of Political Economy, Vol. 81 (1973), pp. 28-55.

Taylor, D.E. “Absences From Work Among Full-time Employees.” Monthly

Labor Review, Vol. 104 (1981), pp. 68-70.

Taylor, P.A. “Income Inequality in the Federal Civilian Government.”

American Sociological Review, Vol. 44 (1979), pp. 468-479.

Taylor, S.H. “The Case for Comparable Worth.” Journal of Social Issues,

Vol. 45 (1989), pp. 23-37.

The Factor Evaluation System of Position Classification Introduction. U.S.

Civil Service Commission, Bureau of Standards of Policies. Washington,

D.C.: 1976.

Thomas, C. “Pay Equity and Comparable Worth.” Labor and Law Journal,

Vol. 34 (1983), pp. 3-12.

Page 133 GAO/GGD-96-20 Federal Job Classification
Bibliography

Thomas, P.J. “Appraising the Performance of Women: Gender and the

Naval Office,” Women’s Career Development. Newbury Park, CA: Sage,

1987.

Thompkins, J. “Sources of Measurement Error and Gender Bias in Job

Evaluation.” Review of Public Personnel Administration, Vol. 9 (1988), pp.

1-16.

Thompson, B.W. National Longitudinal Study of the High School Class of

1977. U.S. Government Printing Office. Washington, D.C.: 1974.

Thomsen, D.J. “Eliminating Pay Discrimination Caused by Job

Evaluation.” Personnel, Vol. 55 (1978), pp. 11-22.

Thomsen, D.J. “Compensation and Benefits.” Personnel Journal, Vol. 62

(1983), p. 38.

Thorndike, E.L. Prediction of Vocational Success. The Commonwealth

Fund. New York: 1934.

Thurow, L.C. Generating Inequality: Mechanisms of Distribution in the U.S.

Economy. New York: Basic Books, 1975.

Tienda, M., and V. Ortiz. “Intraindustry Occupational Recomposition and

Gender Inequality in Earnings,” Ingredients for Women’s Employment

Policy. Albany: State University of New York Press, 1987.

Tompkins, J., J. Brown, and J.H. McEwen. “Designing a Comparable Worth

Based Job Evaluation System: Failure of an A Priori Approach.” Public

Personnel Management, Vol. 19 (1990), pp. 31-42.

Toops, H.A. “Some Concepts of Job Families and Their Importance in

Placement.” Educational and Psychological Measurement, Vol. 5 (1945),

pp. 195-216.

Tornow, W.W. and P.R. Pinto. “The Development of a Managerial Job

Taxonomy: A System for Describing, Classifying, and Evaluating Executive

Positions.” Journal of Applied Psychology, Vol. 61 (1976), pp. 410-418.

Tosi, H.L., R.J. House, and M.D. Dunnette, eds. Managerial Motivation and

Compensation. Board of Trustees of Michigan State University. East

Lansing, MI: 1972.

Page 134 GAO/GGD-96-20 Federal Job Classification
Bibliography

Touthey, J.C. “Effects of Additional Women Professionals on Ratings of

Occupational Prestige and Desirability.” Journal of Personality and Social

Psychology, Vol. 29 (1974), pp. 86-89.

Treiman, D.J. Job Evaluation: An Analytic Review. National Academy of

Sciences. Washington, D.C.: 1979.

Treiman, D.J. “Effect of Choice of Factors and Factor Weights in Job

Evaluation,” Comparable Worth and Wage Discrimination: Technical

Possibilities and Political Realities. Philadelphia: Temple University Press,

1984.

Treiman, D.J., and H.I. Hartmann, eds. Women, Work, and Wages: Equal

Pay for Jobs of Equal Value. Washington, D.C.: National Academy Press,

1981.

Treiman, D.J., and K. Terrell. “Women, Work, and Wages—Trends in the

Female Occupation Structure,” Social Indicator Models. Russell Sage

Foundation. New York: 1975.

Trusheim, D., and J. Crouse. “Affects of College Prestige on Men’s

Occupational Status and Income.” Research in Higher Education, Vol. 14

(1981), pp. 283-304.

Tuckman, B.H. “Salary Differences Among University Faculty and Their

Implications for the Future,” Salary Equity: Detecting Sex Bias in Salaries

Among College and University Professors. Lexington, MA: Lexington

Books, 1979.

Turner, W.D. “The Mathematical Basis of the Percent Method of Job

Evaluation.” Personnel, Vol. 25 (1948), pp. 154-160.

Turner, W.D. “Some Precautions in the Use of the Percent Method of Job

Evaluation.” Journal of Applied Psychology, Vol. 33 (1949), pp. 547-552.

Twenhafel, D. 1991 U.S. Senate Employment Practices: A Study of Staff

Salary, Tenure, Demographics and Benefits. Congressional Management

Foundation. Washington, D.C.: 1991.

20 Facts on Women Workers. U.S. Department of Labor, Office of the

Secretary, Women’s Bureau. Washington, D.C.: 1980.

Page 135 GAO/GGD-96-20 Federal Job Classification
Bibliography

Ulrich, C. “Why Companies are Looking at Omnibus Pay Programs.”

Journal of Compensation and Benefits, Vol. 7 (1991), pp. 30-34.

Ungson, G.R., and R.M. Steers. “Motivation and Politics in Executive

Compensation.” Academy of Management Review, Vol. 9 (1984), pp.

313-323.

Valdez, R.L., and B.A. Gutek. “Family Roles: A Help or a Hindrance for

Working Women?” Women’s Career Development. Newbury Park, CA:

Sage, 1987.

van der Burg, R., and N. Schoemaker. “Scoring Women on Their Labor

Chances.” Women’s Studies International Forum, Vol. 8 (1985), pp.

273-278.

van DeVoort, D.M., J.J. McHenry, and N.E. Fried. “A Policy Capturing

Approach to the Valuing of Managerial Jobs: Developing a Standardized,

Computerized Job Grading System.” Presented at the 43rd Annual National

Meeting of the Academy of Management, Dallas, TX, 1983.

Vladeck, J.P. “Equal Access Is Not Enough.” American Bar Association

Journal, Vol. 70 (1984), pp. 16-21.

Viscusi, W.K. “Sex Differences in Worker Quitting.” Review of Economics

and Statistics, Vol. 62 (1980), pp. 388-398.

Viteles, M.S. “A Psychologist Looks at Job Evaluation.” Personnel, Vol. 17

(1941), pp. 165-176.

Von Frank, J.A. “Equal Pay for Comparable Work.” Harvard Civil Rights

Civil Liberties Law Review, Vol. 15 (1980), pp. 475-506.

Walker, C.T. “The Use of Job Evaluation Plans in Salary Administration.”

Personnel, Vol. 41 (1987), pp. 28-31.

Wall, R.E. “Salary Inequities and Differences: One College’s Attempt at

Identification and Adjustment,” Salary Equity Detecting Sex Bias in

Salaries Among College and University Professors. Lexington MA:

Lexington Books, 1979.

Page 136 GAO/GGD-96-20 Federal Job Classification
Bibliography

Wallace, M.J. “Methodology, Research Practice, and Progress in Personnel

and Industrial Relations.” Academy of Management Review, Vol. 8 (1983),

pp. 6-13.

Wallace, M.J., and C.H. Fay. Compensation Theory and Practice, 2nd ed.

Boston: Pws-Kent Publishing Company, 1988.

Wallace, P.A., ed. Equal Employment Opportunity and the AT&T Case.

Cambridge, MA: The MIT Press, 1976.

Wallace, P.A., and A.M. LaMond, eds. Women, Minorities, and Employment

Discrimination. Lexington, MA: Lexington Books, 1977.

Wallston, B.S., and K.E. Grady. “Integrating the Feminist Critique and the

Crisis in Social Psychology: Another Look at Research Methods,” Women,

Gender, and Social Psychology. Hillsdale, NJ: Lawrence Erlbaum

Associates, 1985.

Ward, K.B., and C.W. Mueller. “Sex Differences in Earnings: The Influence

of Industrial Sector Authority Hierarchy, and Human Capital Variables.”

Work and Occupations, Vol. 12 (1985), pp. 437-463.

Warr, P., and G. Parry. “Paid Employment and Women’s Psychological

Well-being.” Psychological Bulletin, Vol. 91 (1982), pp. 498-516.

Wasem, M.R. “The Comparable Worth Theory, a Critical Analysis.” Baylor

Law Review, Vol. 32 (1980), pp. 629-638.

Webb, N.M., et al. “Generalizability of General Education Development

Ratings of Jobs in the United States.” Journal of Applied Psychology, Vol.

66 (1981), pp. 188-192.

Weber, R.P. Basic Content Analysis. Newbury Park, CA: Sage, 1990.

Werwie, D.M. Sex and Pay in the Federal Government: Using Job

Evaluation Systems to Implement Comparable Worth. New York:

Greenwood Press, 1987.

Wesman, E.C. “Unions and Comparable Worth: Progress in the Public

Sector.” Journal of Collective Negotiations, Vol. 17 (1988), pp. 13-26.

Page 137 GAO/GGD-96-20 Federal Job Classification
Bibliography

White, M.C., M.D. Crims, and G.L. Desanctis. “Ratings of Prestige and

Desirability: Effects of Additional Women Entering Selected Business

Occupations.” Personality and Social Psychology Bulletin, Vol. 7 (1981),

pp. 588-592.

White, P.E. Women and Minorities in Science and Engineering: An Update.

National Science Foundation, Report 303. Washington, D.C.: 1992.

Whyte, W.F. Money and Motivation. New York: Harper and Row, 1955.

Wilcox, A.C. A Method of Determining Equitable Pay. PAQ Services

Research Bulletin 0001-384-120181. 1982.

Wilcox, A.C. A Method of Determining Exemption Status. PAQ Services

Research Bulletin 0002-001-062382. 1982.

Willborn, S.L. A Secretary and a Cook: Challenging Women’s Wage in the

Courts of the United States and Great Britain. Ithica, NY: ILR Press, 1989.

Willborm, S.C. A Comparable Worth Primer. Lexington, MA: Lexington

Books, 1986.

Williams, C.L. Gender Differences at Work: Women and Men in

Nontraditional Occupations. Berkeley, CA: University of California Press,

1989.

Williams, D.R. and C.A. Register. “Regional Variations in Earnings and the

Gender Composition of Employment: Is ‘Women’s Work’ Undervalued?”

Journal of Economic Issues, Vol. 20 (1986), pp. 1121-1134.

Williams, M.L., and G.F. Dreher. “Compensation System Attributes and

Applicant Pool Characteristics.” Academy of Management Journal, Vol. 35

(1992), pp. 571-595.

Williams, R.E., and L.L. Kessler. A Closer Look at Comparable Worth: A

Study of the Basic Questions to be Addressed in Approaching Pay Equity.

National Foundation for the Study of Equal Employment Policy.

Washington, D.C.: 1984.

Willis, N.D. Comparable Worth Study: State of Washington. Norman D.

Willis and Associates. 1974.

Page 138 GAO/GGD-96-20 Federal Job Classification
Bibliography

Wilson, M.A., and R.J. Harvey. “The Value of Relative-time-spent Ratings in

Task-oriented Job Analysis.” Journal of Business and Psychology, Vol. 4

(1990), pp. 453-461.

Wilson, M.A., R.J. Harvey, and B.A. Macy. “Repeating Items to Estimate the

Test-retest Reliability of Task Inventory Ratings.” Journal of Applied

Psychology, Vol. 75 (1990), pp. 158-163.

Wise, D.A. “Academic Achievement and Job Performance.” American

Economic Review, Vol. 65 (1975), pp. 350-366.

Wisniewski, S.C. “Achieving Equal Pay for Comparable Worth Through

Arbitration.” Employee Relations Law Journal, Vol. 8 (1982), pp. 236-255.

Wittig, M.A., and S.L. Berman. “A Set of Validity Criteria for Modeling

Job-based Compensation Systems.” Human Resource Management

Review, Vol. 1 (1991), pp. 107-118.

Wittig, M.A., and R.H. Lowe. “Comparable Worth Theory and Policy.”

Journal of Social Issues, Vol. 45 (1989), pp. 1-21.

Wolf, W.Z., and N.D. Fligstein. “Sex and Authority in the Work Place: The

Causes of Sexual Inequality.” American Psychologist, Vol. 44 (1979), pp.

235-252.

Wolins, L. “Bias in Rating, Unit of Analysis Mistake and Editorial

Decisions.” Manuscript, Iowa State University, 1991.

Women and Minorities in Science and Engineering. National Science

Foundation. Washington, D.C.: 1982.

Women in Management. U.S. Department of Labor, Women’s Bureau.

Washington, D.C.: 1980.

Wong, M.G., and C. Hirschman. “Labor Force Participation and

Socioeconomic Attainment of Asian-American Women.” Sociological

Perspectives, Vol. 26 (1983), pp. 423-446.

Yanico, B.J., and S.I. Hardin. “Sex-role Self-concept and Persistance in a

Traditional vs. Nontraditional College Major for Women.” Journal of

Vocational Behavior, Vol. 18 (1981), pp. 219-227.

Page 139 GAO/GGD-96-20 Federal Job Classification
Bibliography

Zabalza, A. and Z. Tzammatos. Women and Equal Pay: The Effects of

Legislation on Female Employment and Wages in Britain. Cambridge:

Cambridge University Press, 1985.

Zanna, M.P., F. Crosby, and G. Loewenstein. “Male Reference Groups and

Discontent Among Female Professionals,” Women’s Career Development.

Newbury Park, CA: Sage, 1987.

Zedeck, S., and K.L. Mosier. “Work in the Family and Employing

Organization.” American Psychologist, Vol. 45 (1990), pp. 240-251.

Ziering, B.A. and N.S. Raju. “Development and Validation of a Job Family

Specific Position Analysis Questionnaire.” Journal of Business and

Psychology, Vol. 2 (1988), pp. 228-238.

Zippo, M. “Equal Pay for Comparable Work.” Personnel, Vol. 58 (1981), pp.

4-10.

Zuckerman, H., J.R. Cole, and J.T. Bruer, eds. The Outer Circle: Women in

the Scientific Community. New York: W.W. Norton and Company, 1991.

Page 140 GAO/GGD-96-20 Federal Job Classification
Bibliography

Page 141 GAO/GGD-96-20 Federal Job Classification
Bibliography

Page 142 GAO/GGD-96-20 Federal Job Classification
Bibliography

Page 143 GAO/GGD-96-20 Federal Job Classification
Related GAO Products

Pay Equity: Experiences of Canada and the Province of Ontario

(GAO/GGD-94-27BR, Nov. 2, 1993).

Pay Equity: Washington State’s Efforts to Address Comparable Worth

(GAO/GGD-92-87BR, July 1, 1992).

Federal Pay: Comparisons With the Private Sector by Job and Locality

(GAO/GGD-90-81FS, May 15, 1990).

State Department: Minorities and Women are Underrepresented in the

Foreign Service (GAO/NSIAD-89-146, June 26, 1989).

Pay Equity: Status of State Activities (GAO/GGD-86-141BR, Sept. 19, 1986).

Description of Selected Nonfederal Job Evaluation Systems (GAO/GGD-85-57,

July 31, 1985).

Comments on Report on Comparable Worth by the United States

Commission on Civil Rights (GAO/GGD-85-59, June 14, 1985).

Options for Conducting a Pay Equity Study of Federal Pay and

Classification (GAO/GGD-85-37, Mar. 1, 1985).

Distribution of Male and Female Employees in Four Federal Classification

Systems (GAO/GGD-85-20, Nov. 27, 1984).

Description of Selected Systems for Classifying Federal Civilian Positions

and Personnel (GAO/GGD-84-90, July 13, 1984).

Classification of Federal White-Collar Jobs Should Be Better Controlled

(GAO/FPCD-75-173, Dec. 4, 1975).

(966406) Page 144 GAO/GGD-96-20 Federal Job Classification
Ordering Information

The first copy of each GAO report and testimony is free.

Additional copies are $2 each. Orders should be sent to the

following address, accompanied by a check or money order

made out to the Superintendent of Documents, when

necessary. Orders for 100 or more copies to be mailed to a

single address are discounted 25 percent.

Orders by mail:

U.S. General Accounting Office

P.O. Box 6015

Gaithersburg, MD 20884-6015

or visit:

Room 1100

700 4th St. NW (corner of 4th and G Sts. NW)

U.S. General Accounting Office

Washington, DC

Orders may also be placed by calling (202) 512-6000 

or by using fax number (301) 258-4066, or TDD (301) 413-0006.

Each day, GAO issues a list of newly available reports and

testimony. To receive facsimile copies of the daily list or any

list from the past 30 days, please call (202) 512-6000 using a

touchtone phone. A recorded menu will provide information on

how to obtain these lists.

For information on how to access GAO reports on the INTERNET,

send an e-mail message with "info" in the body to:

info@www.gao.gov

PRINTED ON RECYCLED PAPER
United States

General Accounting Office

Washington, D.C. 20548-0001

Official Business

Penalty for Private Use $300

Address Correction Requested

Bulk Rate

Postage & Fees Paid

GAO

Permit No. G100