United States General Accounting Office

GAO Report to the Chairman, Subcommittee

on Oversight and Investigations,

Committee on Economic and

Educational Opportunities, House of

Representatives

August 1995 SCHOOLS AND

WORKPLACES

An Overview of

Successful and

Unsuccessful Practices

GAO/PEMD-95-28

GAO United States

General Accounting Office

Washington, D.C. 20548

Program Evaluation and

Methodology Division

B-262061 

August 31, 1995

The Honorable Pete Hoekstra

Chairman, Subcommittee on Oversight

 and Investigations

Committee on Economic and Educational

 Opportunities

House of Representatives

Dear Mr. Chairman:

The nation’s well-being depends on its ability to create and sustain

well-paying jobs and to improve the performance of U.S. business in an

increasingly complex world economy. For more than a decade, Americans

have been concerned that the nation is not doing all that is needed to meet

these challenges. They have focused this concern especially on the quality

of education provided by elementary and secondary schools (particularly

those attended by disadvantaged students) and on the productivity and

performance of workers and their employers. You asked us to help you

address these issues by summarizing research findings on what has and

what has not been successful in schools and workplaces.

Objectives, Scope,

and Methodology

To provide the information you requested, we reviewed a large number of

reviews and studies concerning successful organizations (elementary and

secondary schools and firms in a wide range of businesses) and successful

practices, as well as the relatively few studies on practices that do not

work. We focused on practices at the organizations’ sites that shape the

work environment and human resources within the school or business

workplace, and we also examined general curricular and instructional

practices in schools and general management processes in the business

workplace. We did not examine how practices are shaped by public policy,

nor did we attempt to cover specific instructional or business practices

(such as particular methods of teaching reading or managing inventory).

Annotated bibliographies appear in appendix I (on schools) and appendix

II (on workplaces).

We relied on the most recent reviews and studies available. With few

exceptions, the materials we examined were published in 1990 or later. We

identified the reviews and studies by searching compendia compiled by

relevant federal agencies (the Office of Educational Research and

Improvement and the Planning and Evaluation Service, both in the

Page 1 GAO/PEMD-95-28 Schools and Workplaces
B-262061 

Department of Education, and the Office on the American Workplace in

the Department of Labor); peer-reviewed professional journals in

education, management, and industrial relations; and citation indexes.

Given the normal limitations on such searches and the relatively short

time period for meeting the subcommittee’s needs, we cannot be sure that

we identified every relevant publication. However, we believe that our

review covers most of the major issues and practices for which evidence is

available.

We searched for studies linking school or workplace practices to

particular, commonly used criteria of success. For schools, these criteria

were (1) student achievement at or above the expected level, (2) high

teacher and student engagement in learning activities, and

(3) effectiveness in overcoming disadvantage, such that students who are

behind when they enter school are able to catch up. The criteria for

workplace success were (1) increased profitability, (2) improved

productivity, and (3) high performance in the workplace.

We selected studies that used valid and reliable measures of both the

success criteria and the implementation of particular practices. We gave

preference to findings that could be generalized to other schools and

workplaces. For some of the success criteria, however, few studies met all

these standards. Therefore, we included studies that had technical

limitations (such as small sample size, low response rates, or surrogate

outcome measures) and even some that simply summarized experts’

conclusions without providing supporting data. The annotations in the

bibliographies describe these limitations.

In using the bibliography, readers should bear in mind that the studies are

of uneven quality and that every study design has its strengths and

weaknesses, so that findings from a single study should be interpreted

with caution. For example, nationally representative sample surveys can

provide reliable estimates of how widely a practice is used and how

strongly it is associated with an indicator of school or business success,

but they typically cannot establish a causal link between a practice and its

outcome. Case studies of one firm or type of school can provide insights

into how a practice works in a specific context, but their findings cannot

necessarily be extended to businesses or schools generally.

Given the limited time available and the diversity of measures and designs

used in the studies, we did not attempt a meta-analysis. Rather, we drew

from the studies what we judged to be the most significant and consistent

Page 2 GAO/PEMD-95-28 Schools and Workplaces
B-262061 

findings—findings supported by a variety of sources—to highlight in this

report. We conducted this review from March through June 1995 in

accordance with generally accepted government auditing standards.

Summary Successful schools and company workplaces had a well-defined mission:

they knew what they wanted to accomplish. They organized the work

environment, human resource practices, and curriculum and instruction or

management processes in ways that supported that mission. The evidence

that we reviewed consistently demonstrated the need to integrate mission

and key practices. No one practice, by itself, seems to ensure success.

The missions in schools were focused on student learning. Work

environment practices consistent with this included providing safe and

orderly sites, encouraging parents’ involvement and collaboration among

staff, fostering leadership for instructional improvement, and authorizing

school-level problem solving. Human resource practices included

school-based control over entry and exit of staff and students and the

encouragement of professional development. In terms of curriculum and

instruction, successful schools established academically rigorous and

well-focused curricula, provided effective and engaging instruction,

exposed all students to challenging curricula and instruction, and ensured

that students who needed extra assistance were given opportunities for

success.

In company workplaces, successful work environments typically

developed a set of core organizational values that were transmitted to all

employees, fostering a sense of community throughout the organization

and encouraging meaningful employee participation in work-related

decisions. In addition, successful companies tended to adopt human

resource policies that featured minimizing job disruption and providing

education and training programs for employees, widely available

profit-sharing and gain-sharing plans, and a reward structure that was

perceived as fair and understandable. Also, successful companies that

went through downsizing or otherwise made significant changes in

company operations made efforts to retrain employees or provide

assistance in obtaining other employment.

We have previously reported on organizational culture and human

resource issues such as managing workforce reductions and using

employee stock ownership plans. In addition, we have reported on a wide

range of management practices such as total quality management,

Page 3 GAO/PEMD-95-28 Schools and Workplaces
B-262061 

organizational reengineering, and other innovations such as benchmarking

and performance measurement. Abstracts of these reports are in appendix

II.

Agency Comments Responsible officials from the U.S. Department of Education and the U.S.

Department of Labor provided written comments on a draft of this report.

Officials from both departments agreed that our findings are consistent

with current knowledge of effective practices in schools and workplaces.

As we arranged with your office, unless you publicly announce the report’s

contents earlier, we plan no further distribution of it until 30 days after the

date of this letter. We will then send copies of this report to the Secretary

of Education, the Secretary of Labor, and others who are interested. We

will also make copies available to others on request. If you have any

questions or would like additional information, please call me at

(202) 512-2900. Other major contributors to this report are listed in

appendix III.

Sincerely yours,

Joseph F. Delfico

Acting Assistant Comptroller General

Page 4 GAO/PEMD-95-28 Schools and Workplaces
Page 5 GAO/PEMD-95-28 Schools and Workplaces
Contents

Letter 1

Appendix I 

Schools: Annotated

Bibliography

8

Work Environment 8

Human Resources 21

Curriculum and Instruction 33

Appendix II 

Workplaces:

Annotated

Bibliography

49

Work Environment 49

Human Resources 70

Management Processes 86

Appendix III 

GAO Contacts and

Staff

Acknowledgments

98

Related GAO Products 100

Abbreviations

DOL U.S. Department of Labor

ESOP Employee stock ownership plan

GAO U.S. General Accounting Office

SAT Scholastic Achievement Test

Page 6 GAO/PEMD-95-28 Schools and Workplaces
Page 7 GAO/PEMD-95-28 Schools and Workplaces
Appendix I 

Schools: Annotated Bibliography

Work Environment

Bryk, Lee, and Holland Anthony S. Bryk, Valerie E. Lee, and Peter B. Holland. Catholic Schools

and the Common Good. Cambridge, Mass.: Harvard University Press, 1993.

Setting Catholic secondary schools that enroll mainstream students, including

disadvantaged students.

Source of Information Quantitative analyses of national data on public and Catholic and other

private schools from the High School and Beyond studies of students,

teachers, and administrators. Case studies of seven diverse Catholic high

schools in the early 1980s with a brief update to the early 1990s.

Findings and Conclusions Features distinguishing Catholic from public high schools include smaller

size, academic focus, and inclusion of all students in the core academic

curriculum. Curricula are standard academic fare and teaching is largely

“ordinary.” Nonetheless, student engagement, satisfaction, and

achievement exceed those in public schools, especially for disadvantaged

students. Student self-selection and greater participation in academic

courses account for some of the difference. But communal organization

(shared values, shared academic and other activities, engagement between

children and adults, and collegial interaction) is also critical.

Catholic schools also exhibit greater autonomy than public schools,

“effective” organizational practices such as principal’s leadership and

faculty professionalism, and responsiveness to the private education

market. However, data from non-Catholic private schools suggest that

these organizational features by themselves, without the kinds of values

and social equity aims that undergird Catholic schooling, do not produce

the higher achievement and lower dropout rates of at-risk youths in

Catholic schools.

Comments Students (disadvantaged or otherwise) in Catholic high schools are

somewhat academically oriented. It is not known whether the Catholic

school approach would work equally well in schools with high

concentrations of disadvantaged students.

Page 8 GAO/PEMD-95-28 Schools and Workplaces
Appendix I 

Schools: Annotated Bibliography

Work Environment

Bryk, Lee, and Smith Anthony S. Bryk, Valerie E. Lee, and Julia B. Smith. “High School

Organization and Its Effects on Teachers and Students: An Interpretative

Summary of the Research.” In William H. Clune and John F. Witte (eds.).

Choice and Control in American Education. Vol. 1. The Theory of Choice

and Control in Education. New York: The Falmer Press, 1990.

Setting High schools in general.

Source of Information Empirical and analytical literature on high school organization and

curricula and their effect on teachers and students.

Findings and Conclusions School features and practices that foster teacher effort and satisfaction

and student engagement and learning include

1. Size large enough to offer adequate instruction but not so large as to

impede social relations.

2. A nucleus of motivated and academically able children to provide a

stable institutional base and avoid discipline problems and an

anti-academic peer culture.

3. Parents who provide high expectations and home support for learning.

4. Management control over student and faculty entry, socialization, and

exit.

5. Management that facilitates informal social interactions, buffers the

core work of the school against external disruptions, communicates

information and allocates resources efficiently and informally, administers

rules justly and fairly, and monitors staff and directs its development to

coincide with teacher goals and foster collegiality.

6. Teachers who influence students’ social and personal development, not

just academic learning.

7. A cooperative work ethic and collaboration among staff.

8. A curriculum in which all students—not just the most able—take

academic courses and receive stimulating instruction.

Page 9 GAO/PEMD-95-28 Schools and Workplaces
Appendix I 

Schools: Annotated Bibliography

Work Environment

Hanushek Eric Hanushek. Making Schools Work: Improving Performance and

Controlling Costs. Washington, D.C.: The Brookings Institution, 1994.

Practice School governance and management; teacher certification, pay systems,

and training.

Source of Information Literature reviews by economists with a broad spectrum of views who are

long-term students of education.

Findings and Conclusions Conventional wisdom holds that education can be improved by reducing

class size, lengthening the school year, or hiring teachers who have more

education and experience. The evidence does not bear this out.

Performance incentives may improve productivity, but designing workable

incentive systems has proven elusive. For example, the success of

performance contracting—contracting with an independent firm for

specified achievement results—depends on writing a good contract, which

is difficult to do. Merit pay for teachers is also difficult to implement. Most

attempts have been underfinanced and have boiled down to extra pay for

extra duties.

School-based management is often seen as an end in itself rather than as a

way of improving student performance. Many school-based management

programs have lacked clear goals, performance improvement incentives,

and accountability mechanisms. Few programs have been evaluated in

terms of performance, and few evaluations have shown achievement

gains.

Choice plans take a variety of forms, from magnet schools to tuition tax

credits or vouchers. Their incentives vary greatly. The mixed evidence

indicates that choice does not necessarily lead to change. Higher

education provides some lessons: poor and mediocre schools continue

despite choice and competition. And vouchers or extra payments may be

needed to ensure that the disadvantaged are served.

Comments The book presents general conclusions, referring to source documents for

supporting data.

Page 10 GAO/PEMD-95-28 Schools and Workplaces
Appendix I 

Schools: Annotated Bibliography

Work Environment

Hill, Foster, and Gendler Paul T. Hill, Gail E. Foster, and Tamar Gendler. High Schools with

Character. Santa Monica, Calif.: The RAND Corporation, August 1990.

Setting Urban public and Catholic high schools attended by low-income minority

students.

Source of Information Intensive case studies of three Catholic schools, three special purpose

public schools, and two comprehensive (zoned) public schools, with

less-intensive studies of five additional public schools (including two in

Washington, D.C.). Hill and colleagues examined school demographics,

mission, and organizational strength and student perceptions, attitudes,

and outcomes (graduation rates, percentage graduating with Regents

diploma, percentage taking the Scholastic Achievement Test (SAT),

percentage of blacks scoring above the national average for black students

on the SAT).

Findings and Conclusions The Catholic schools and special-purpose public schools have important

features in common: a clear, uncomplicated mission, culture, and core

curriculum; a focus on student outcomes; strong internal organization; and

the capacity to solve their own problems and manage their external

relationships. Outcomes for disadvantaged students in these “focus”

schools are far better than for the zoned schools, which have diffuse

missions and fragmented curricula established by central authorities,

emphasize program delivery rather than outcomes, and have relatively

little capacity for self-definition and self-governance. Students and staff

are dissatisfied with the zoned schools. Most zoned-school students could

benefit from attending focused schools, either schools oriented toward the

mainstream student or those designed to meet special needs.

Comments The zoned schools enroll much higher concentrations of educationally

disadvantaged students than the focus schools, and teachers consider

transience a problem. The “mainstream” focus schools might not operate

as successfully if they enrolled the same cross-section of students as the

zoned schools. Outcome measures are limited; the SAT is a proxy for

students considering applying to selective colleges.

Page 11 GAO/PEMD-95-28 Schools and Workplaces
Appendix I 

Schools: Annotated Bibliography

Work Environment

Kelly Dierdre Kelly. Last Chance High: How Girls and Boys Drop in and Out of

Alternative Schools. New Haven, Conn.: Yale University Press, 1993.

Practice The use of continuing high schools to prevent students from dropping out.

Setting Continuing high schools in California.

Source of Information Two case studies and a synthesis of research literature.

Findings and Conclusions Continuing high schools, sometimes called alternative or second-chance

schools, are the most common dropout prevention program in the United

States. A safety net for some students, they have not been effective in

retaining students until graduation. In California, where 104,000 students

attended continuing high schools in 1986-87, only 10 percent have received

a diploma or its equivalent.

Continuing schools enroll students whose needs and abilities diverge from

those of students who engage in regular high school activities. Students

who choose to enroll tend to have higher completion rates than those who

are coerced into enrolling. A stigma is often attached to continuing

schools that have been assigned disciplinary cases, limit reengagement,

and push students out.

Comments The author notes that the continuing schools she studied lacked a

well-developed vision, used a remedial curriculum, and did not prepare

students for specific occupations. Because of the small number of cases,

her findings cannot be generalized, nor do they suggest much about

schools that do not share these characteristics.

Page 12 GAO/PEMD-95-28 Schools and Workplaces
Appendix I 

Schools: Annotated Bibliography

Work Environment

Laguarda et al. Katrina G. Laguarda et al. “Raising the Educational Achievement of

Secondary School Students: An Idea Book,” U.S. Department of Education,

Washington, D.C., 1995.

Practice Describes practices to strengthen and enrich the curriculum, create

communities of learners, link school work to students’ futures, and create

networks of support for students. Also lists resources for school

improvement.

Setting Secondary schools, including magnet schools and charter schools.

Source of Information Volume 1 reflects conclusions drawn from the research literature on

effective secondary schools, curriculum, and instruction. Volume 2

presents profiles of 13 secondary schools, each illustrating one or more of

the described practices and citing evidence of the school’s success.

Findings and Conclusions Secondary schools are most successful when all students, including low

achievers, engage in work that is academically rigorous, challenging, and

makes sense to them. Ideas for increasing engagement include covering

fewer topics but studying them in depth; interdisciplinary approaches;

internships and other forms of service learning; and integrating academic

with occupational preparation. Techniques such as cooperative learning

can help lower-achieving students succeed in the more challenging

coursework of the regular program.

Smaller school size, the creation of smaller units within schools, and

permitting students greater freedom in choosing their school can help

increase community. Flexible use of time can broaden the range of

learning experiences possible.

Peer tutoring and peer or adult mentoring provide students with a sense of

membership and support. Practices that build links to families and social

services are also helpful.

Comments For ease of use, citations to individual research reports and papers were

omitted. References are listed in volume 1.

Page 13 GAO/PEMD-95-28 Schools and Workplaces
Appendix I 

Schools: Annotated Bibliography

Work Environment

Lee and Smith Valerie E. Lee and Julia B. Smith. “Effects of High School Restructuring

and Size on Gains in Achievement and Engagement for Early Secondary

School Students.” University of Wisconsin, Center for Organization and

Restructuring of Schools, Madison, Wisconsin, April 1, 1994.

Practice Traditional organization, moderate reform practices, and “restructuring”

or organizational practices that foster continuity in teacher-student

contacts, cooperative teaching or learning, and staff involvement in

common planning and school problem solving.

Setting Public and Catholic and other private high schools.

Source of Information Eighth- and 10th-grade test scores and other school and student data from

the National Education Longitudinal Study of 1988. Analysis covers nearly

12,000 students in 801 schools.

Findings and Conclusions Small size, academic emphasis, and restructuring are associated with

greater student engagement and gains in learning. The distribution of

engagement and learning gains across students of different social

backgrounds is more equitable in restructured than in either moderately

reformed or traditionally organized high schools, which are larger and

more internally stratified than the other types and enroll higher

proportions of low-income and minority students.

Schools get best results when they build a strong core of operation by

adopting only a few restructuring reforms—along with such conventional

reforms as emphasizing academic requirements, parent involvement, and

staff stability. Small size makes both types of reform easier to implement.

Comments The scores used to measure growth in student achievement were not

highly reliable, and the survey data comprised relatively rough indicators

of student engagement and school practice. The authors cite case study

research as evidence that their findings are on target despite these

limitations.

Page 14 GAO/PEMD-95-28 Schools and Workplaces
Appendix I 

Schools: Annotated Bibliography

Work Environment

Levine and Ornstein Daniel U. Levine and Allan C. Ornstein. “Research on Classroom and

School Effectiveness and Its Implications for Improving Big City Schools.”

The Urban Review, 21:2 (1989), 81-95.

Setting Elementary schools enrolling high proportions of disadvantaged children.

Source of Information Literature on effective schools and teaching.

Findings and Conclusions Unusually effective elementary schools for disadvantaged students are

characterized by (1) an orderly environment, (2) a clear mission, (3) the

principal’s instructional leadership, (4) a climate of high expectations,

(5) high time-on-task, (6) frequent monitoring of student progress, and

(7) positive home-school relationships. Effective teaching for such

students combines direct instruction (aimed at building basic skills) with

techniques designed to elicit participation and facilitate comprehension

and higher-order thinking.

Efforts to apply the organizational principles of effective schools to

improve other schools have met with mixed success. It is useful for

schools seeking improvement to (1) focus on a few clearly defined

instructional goals, (2) provide technical assistance and time for staff

development, (3) be guided by data and by approaches that have

succeeded elsewhere, (4) practice “vigorous selection and replacement of

teachers,” and (5) be flexible in their approach and maintain a judicious

mixture of faculty autonomy and central office directiveness.

Comments This article usefully summarizes the principles of effective schools for a

general audience. These principles were derived from research on

inner-city elementary schools, and it may not be possible to generalize to

other settings (such as secondary schools or schools that serve students

from middle-income families).

Page 15 GAO/PEMD-95-28 Schools and Workplaces
Appendix I 

Schools: Annotated Bibliography

Work Environment

Mohrman et al. Susan A. Mohrman et al. School-Based Management: Organizing for High

Performance. San Francisco, Calif.: Jossey-Bass, 1994.

Practice School-based management.

Source of Information Reviews of program materials and research literature.

Findings and Conclusions School-based management is a popular reform that ranges from electing

school governing boards to enhancing decisionmaking authority for

principals or teachers. In some places, it has improved classroom practice

and student achievement. Overall, however, evidence of its efficacy is not

compelling.

Typically, school-based management programs and the concepts behind

them are vaguely described and focus on changing governance, not on

improving school outcomes. Although the programs decentralize

decisionmaking power, most do not provide a school’s participants with

the knowledge, skills, and other resources to enable them to improve

teaching and learning.

Creating new governance structures, by itself, is not enough. School-based

management must also be linked to improving a school’s goals and

processes. Both the district’s and the school’s roles in these processes

must be clearly defined, real decisionmaking authority must devolve to the

school, and administrators’ roles must be changed accordingly. It is useful

to employ a variety of mechanisms (not just a governing council) to

involve teachers, parents, and others in the school’s management and

improvement. A school’s participants should have access to information

about it and its performance and about performance and practice in other

schools. School-based management is more likely to help improve

instruction if the school’s participants are held accountable for outcomes

and rewarded for improving school performance.

Comments Information about existing programs is too sketchy to sustain conclusions

about cause and effect. The authors’ conclusions reflect a conceptual

analysis of school-based management program design, informed by

knowledge of school improvement processes and high-performance

businesses.

Page 16 GAO/PEMD-95-28 Schools and Workplaces
Appendix I 

Schools: Annotated Bibliography

Work Environment

Pauly Edward Pauly. The Classroom Crucible: What Really Works, What Doesn’t,

and Why. New York: HarperCollins, 1991.

Setting Mainstream elementary and secondary schools.

Source of Information Site visits and interviews with school principals, teachers, students, and

parents in schools in California, Connecticut, Massachusetts, New York,

Pennsylvania, and Virginia. Review of relevant research.

Findings and Conclusions Successfully implementing programs depends on

1. devising new classroom membership policies, including improved

attendance, team teaching, and increased parent and student choice;

2. introducing new classroom support policies, including principals’

responses to unique classroom problems;

3. increasing teachers’ salaries to enlarge and improve the pool of

potential teachers;

4. stabilizing classroom membership by providing transportation subsidies

for students who move during the school year, providing 2-hour high

school classes, and maintaining the same classroom membership for 2

school years;

5. creating alternative schools that recruit teachers and students who wish

to work in nontraditional settings;

6. increasing outreach activities to encourage dropouts to return to school

by, for example, contracting with community agencies to recruit and

support them.

Successful innovations often cannot be replicated in other settings: people

tend to work harder when they think they are part of a new, important

activity, but many effects tend to fade away over time. Teachers alter and

reformulate the innovations that they are supposed to implement.

Comments Findings are based primarily on observational data but are supported by

external educational research evidence.

Page 17 GAO/PEMD-95-28 Schools and Workplaces
Appendix I 

Schools: Annotated Bibliography

Work Environment

Raywid Mary Ann Raywid. “Alternative Schools: The State of the Art.” Educational

Leadership, 52:1 (September 1994), 26-31.

Practice Alternative organizational structures and instructional strategies.

Setting Secondary schools for students who have not fared well in conventional

schools.

Source of Information Synthesis of case study literature.

Findings and Conclusions Alternative schools that succeed with students who do not prosper in

conventional settings have in common that they (1) generate and sustain

community and are caring places that feel like family, (2) make learning

engaging, and (3) provide school organization and structure to sustain

community and engagement. All three factors must be present: no one

factor is effective without the others.

Successful alternative schools tend to be small, designed and led by the

people who operate them, and reflective of those teachers’ strengths and

interests. Newly hired teachers choose and are chosen by a school, which

operates with few specialized staff. Students and families choose the

school as well. Such schools operate relatively autonomously. Curricular

thrusts and instructional strategies vary, but each school offers a

challenging, inviting learning environment and opportunities for students

to succeed.

Comments This article presents general lessons learned from case studies, drawing

particularly from studies of schools in Community District 4 in New York

City. It does not present data on student outcomes. The author is a leading

authority on schools of choice.

Page 18 GAO/PEMD-95-28 Schools and Workplaces
Appendix I 

Schools: Annotated Bibliography

Work Environment

Rosenholtz Susan J. Rosenholtz. Teachers’ Workplace: The Social Organization of

Schools. New York: Longman, 1989.

Practice School goals and work environments.

Setting Elementary schools in Tennessee.

Source of Information Survey data from 1,213 teachers in 78 elementary schools in 5 rural and 3

urban or suburban districts in Tennessee; school demographic,

attendance, and achievement data; interviews with 74 teachers from 23

schools that were either very high or very low on measures of goal

consensus and collaboration.

Findings and Conclusions Schools differ markedly along the dimensions of (1) goal consensus with

respect to teaching and learning; (2) collaboration among teachers;

(3) teachers’ learning opportunities; (4) their conceptions of teaching; and

(5) teacher commitment. These differences reflect organizational practices

of teacher socialization, evaluation, and involvement in decisionmaking

and the goal-setting, monitoring, and support activities of principals and

district officials. They are reflected in teachers’ talk about students and

teaching, leadership and mutual assistance among teachers, their efforts to

increase their professional skills, and their conception of teaching as

either a routine technology or one that can vary to meet individual

students’ needs.

Schools characterized by goal consensus, collaboration, and professional

growth differ from schools where teachers have no common goal, are

isolated from one another, and are discouraged from asking for help from

colleagues or the principal. Isolated teachers become “stuck” teaching as

they have always taught. They have less motivation to achieve than

teachers in settings that offer greater professional fulfillment. Teachers’

learning opportunities and commitment are positively associated with

student achievement.

Comments Some of the quantitative findings are based on small sample sizes and

should be treated with caution. Quotations from teacher interviews

graphically illustrate the differences in work environment and conceptions

of teaching across schools.

Page 19 GAO/PEMD-95-28 Schools and Workplaces
Appendix I 

Schools: Annotated Bibliography

Work Environment

Teddlie and Stringfield Charles Teddlie and Sam Stringfield. Schools Make a Difference: Lessons

Learned from a 10-Year Study of School Effects. New York: Teachers

College Press, 1993.

Setting Rural, suburban, and urban public elementary schools in Louisiana, with

students of various socioeconomic status.

Source of Information School, administrator, teacher, and student data—including test

scores—from a representative sample of 76 schools, followed by case

studies of 8 matched pairs of schools (one more effective and one less

effective than predicted from students’ socioeconomic status) over 10

years. Focus on third grade.

Findings and Conclusions School and teacher or classroom factors account for about 25 percent of

the variance in student achievement. Effective schools for students of both

middle and low socioeconomic status exhibit (1) a clear academic mission

and focus, (2) an orderly environment, (3) effective use of instructional

time, and (4) frequent monitoring of student progress. Teachers hold high

expectations for their students and push them to achieve. Supportive

induction of inexperienced teachers helps build and sustain effective

teaching.

Other factors associated with effectiveness differ in different school

contexts. For example, principals of effective schools with students of low

socioeconomic status initiate change and serve as instructional leaders for

their generally less-experienced teaching staff, while principals of effective

schools with students of middle socioeconomic status act as managers,

giving their generally more experienced teachers responsibility for

instructional leadership.

Some schools remained effective over the 10 years (in part by adapting

appropriately to socioeconomic changes) while others declined. Changes

in principals and faculty led to marked improvement in some formerly

ineffective schools, but others remained ineffective.

Comments Extends “effective schools” research by considering a broad range of

contexts, attending to classroom practice, and examining how schools

improve or decline. The case studies illustrate practices that permit poor

as well as good teaching.

Page 20 GAO/PEMD-95-28 Schools and Workplaces
Appendix I 

Schools: Annotated Bibliography

Human Resources

Brandt Richard M. Brandt. Incentive Pay and Career Ladders for Today’s

Teachers. Albany, N.Y.: State University of New York Press, 1990.

Practice Incentive pay and career ladder programs.

Source of Information Literature review and case studies of Arizona, South Carolina, Utah, and

Virginia programs in the 1980s.

Findings and Conclusions Incentive pay and career ladder programs created in many states and

school districts in the mid-1980s were largely designed locally and varied

greatly: they might feature merit pay, career ladders, or elements of both.

Some were mandatory, others voluntary. Relatively few survived for as

long as 5 years. Those that did rewarded many teachers for performing

expanded instructional duties rather than recognizing a few for superior

teaching.

The effort required to plan and implement the programs that survived was

substantial, as was the cost. Their effects on teacher motivation were

mixed. Some programs produced modest gains in student achievement;

others had little apparent effect.

The benefits from these programs stemmed mainly from their

development of more demanding, comprehensive approaches to teacher

evaluation. The tasks of identifying criteria for good teaching and applying

them through classroom observation and assessments of student progress

stimulated teachers’ thinking about, and principals’ involvement in,

instruction.

Had they not embarked on incentive pay systems, school districts would

not have improved their evaluation systems. The pay element itself was

secondary to these improvements as a source of benefits.

Comments Studies have relied primarily on interviews and attitude surveys for

evidence of the effect of incentive programs on teachers and students.

Only a few have examined the connection between teacher pay incentives

and student outcomes. Many variables affect student performance, and the

effects of teacher pay are difficult to isolate.

Page 21 GAO/PEMD-95-28 Schools and Workplaces
Appendix I 

Schools: Annotated Bibliography

Human Resources

Conley and Odden Sharon Conley and Allen Odden. “Linking Teacher Compensation to

Teacher Career Development.” Educational Evaluation and Policy

Analysis, 17:2 (1995), 219-37.

Practice Pay plans based on teacher performance, skill, and knowledge.

Source of Information Review of research literature on teacher motivation and compensation and

case studies of four career ladder programs.

Findings and Conclusions Merit pay and career ladder systems raise doubts about whether an

appropriate alternative to conventional teacher pay systems can be found.

Individual merit or performance-based pay is not a promising alternative.

It works best when workers are independent of one another, the

technology is straightforward, and results are easy to measure. It seems

ill-suited to teaching, which is inherently collegial. Moreover, teachers

seek primarily the intrinsic rewards from helping students learn. Money is

not the primary motivator.

Job-based pay, which reflects differences in the complexity or importance

of the work performed, also has drawbacks, particularly when many

teachers must compete for scarce top-level positions.

Increases in pay can also be linked to gains in skills and knowledge—to

stages in career development. This seems potentially advantageous in that

it is connected to intrinsic as well as extrinsic rewards, does not pit

teachers against one another, and encourages professional growth on the

part of all. Evidence from private industry, higher education, and the few

districts that have tried it suggest that skill-based pay would be beneficial

to teaching.

Changes in compensation systems must be made carefully to ensure that

the new system is compatible with organizational goals, structure, and

norms (such as collegiality).

Comments Information on student achievement was not available.

Page 22 GAO/PEMD-95-28 Schools and Workplaces
Appendix I 

Schools: Annotated Bibliography

Human Resources

Duke Daniel L. Duke. “Removing Barriers to Professional Growth.” Phi Delta

Kappan, 74:9 (1993), 702-12.

Practice Teacher evaluations.

Setting Public and private elementary and secondary schools throughout the

nation.

Source of Information Review of research literature.

Findings and Conclusions Traditional teacher evaluations may be of some use for new and unskilled

teachers in that they can foster accountability by formally documenting a

teacher’s deficiencies and mutual agreements to pursue corrective

strategies. However, traditional teacher evaluations have failed to promote

professional growth and development.

For teachers in need of assistance, evaluations may be perceived as

negative, adversarial, and threatening, and they can thwart improvement.

Administrators typically have to conduct so many routine evaluations that

they have little opportunity to invest in cooperative and sustained efforts

to help teachers improve classroom performance.

Most experienced teachers meet or exceed the standards upon which they

are rated. Evaluations of them reinforce their continuation of ordinary

practices and fail to make them aware of the need for continual

professional growth.

Evaluations for accountability should be separate from evaluations for

professional growth, and the latter should be emphasized for experienced

and competent teachers.

Comments Conventional teacher evaluations consist of a set of performance

standards and a series of categories by which these standards can be

rated. Administrators observe teachers several times a year and then

annually summarize the information gathered from their observations.

Professional growth plans, in contrast, reflect progress toward individual

or organizational goals, with this progress evaluated by both

administrators and teachers.

Page 23 GAO/PEMD-95-28 Schools and Workplaces
Appendix I 

Schools: Annotated Bibliography

Human Resources

Firestone and Pennel William A. Firestone and James R. Pennel. “Teacher Commitment,

Working Conditions, and Differential Incentive Policies.” Review of

Educational Research, 63:4 (1993), 489-525.

Practice Differential incentive policies, including merit pay and career ladders.

Setting Public and private schools.

Source of Information Review of qualitative and quantitative studies of teachers’ commitment,

working conditions, and incentives.

Findings and Conclusions Working conditions, including incentive systems, shape the teachers’

commitment to the school, to teaching, and to students. Committed

teachers are motivated to change instructional practices and maintain a

drive for excellence.

Commitment is enhanced by meaningful and varied work, participation in

decisionmaking, feedback, collaboration, learning opportunities, an

orderly work environment, administrative support, a reasonable workload,

a suitable physical environment, appropriate instructional resources,

adequate pay, and job security.

Differential incentive plans have had mixed effects. Merit pay and other

programs that attempt to link pay and individual performance raise

concerns about fairness and may reduce collaboration and diminish

learning opportunities.

Career-ladder and mentoring programs that mix financial and intrinsic

rewards and have weaker competitive aspects enhance commitment by

increasing collaboration, participation, and learning opportunities.

Comments Details of individual differential incentive plans vary greatly. This makes it

difficult to evaluate the effect of these plans collectively.

Page 24 GAO/PEMD-95-28 Schools and Workplaces
Appendix I 

Schools: Annotated Bibliography

Human Resources

Frase Larry Frase (ed). Teacher Compensation and Motivation. Lancaster, Pa.:

Technomic Publishing Company, 1992.

Practice Teacher incentive pay and career ladder programs.

Setting Public and private elementary and secondary schools.

Source of Information Literature on motivation in schools and workplaces; case studies of state

and local public school incentive pay and career ladder programs and

private school pay systems.

Findings and Conclusions Some local school districts have developed incentive pay and career

ladder programs that, according to superintendents, have increased

professionalism and encouraged marginally qualified teachers to depart.

Reviews of pilot programs in several states point to similar perceived

benefits when they enhanced professional opportunities rather than

simply raising pay.

Incentive programs, however, are costly. Many have been ill-designed for

the context of teaching and soon abandoned. (Programs restricted to

awarding extra pay to a few “outstanding” teachers are a case in point.)

State programs have been implemented unevenly and have departed little

from conventional practice in some places.

Conclusions about success have been based largely on attitudinal data

from teachers and administrators, not on performance measures. Overall,

the effect of incentive pay on teacher productivity and performance is

unclear.

The element of these programs most popular with legislators—merit

pay—is the most difficult to implement and is weakly related to teacher

motivation. Other elements, such as better teacher evaluation procedures

and expanded opportunities for professional growth, are more likely to

have motivating power. These elements can be instituted within

conventional pay plans.

Comments The book covers only plans directed toward rewarding individual

teachers. School-based incentive programs are too new for their effects to

have been studied.

Page 25 GAO/PEMD-95-28 Schools and Workplaces
Appendix I 

Schools: Annotated Bibliography

Human Resources

Fullan and Stiegelbauer Michael G. Fullan and Suzanne Stiegelbauer. The New Meaning of

Educational Change. New York: Teachers College Press, 1991.

Practice Teacher staff development and teacher evaluation as tools for change.

Source of Information Review of research on policies and practices connected to educational

change.

Findings and Conclusions Instructional innovations significant enough to affect student learning

require complex changes among teachers and in schools. Many

innovations fail because they are technically unsound (fads and quick

fixes) or poorly implemented or because the schools are overloaded with

demands for change.

Staff development is an important tool for implementing innovations.

However, conventional practice does not go far enough to be effective.

Simply equipping teachers with new skills in a workshop setting (the

typical approach) does not ensure that they will apply these skills in the

classroom. They need follow-up support to help link their new skills to

their prior knowledge, try them out, adjust them to their own classroom

circumstances, review the results with colleagues, and adjust them

again—in short, to help them learn by experience in a supportive

environment.

Teacher evaluation systems that are pro forma (as is common) are a waste

of time. Evaluation systems that are oriented toward teacher

accountability are costly and yield few benefits. Both typically reflect

minimal competency or “effective teaching behavior” rather than

improving teachers’ skills. Exemplary systems oriented toward

professional development foster growth and change among capable

teachers. Under such systems, teachers who receive low ratings for

several consecutive years are likely to leave the profession.

Comments The author evaluates each practice in terms of its potential for

successfully implementing needed change. The practices may serve other

purposes as well.

Page 26 GAO/PEMD-95-28 Schools and Workplaces
Appendix I 

Schools: Annotated Bibliography

Human Resources

Johnson Susan Moore Johnson. Teachers at Work. New York: Basic Books, 1990.

Setting Public, Catholic, and independent elementary and secondary schools.

Source of Information Interview with a purposive sample of 115 top-rated teachers who work in

economically and demographically diverse schools.

Findings and Conclusions Intrinsic and extrinsic rewards promote good teaching. Schools that

attract and retain well-educated and highly motivated teachers are

focused, foster a sense of belongingness, and offer substantial

psychological rewards.

However, adequate pay, opportunities for promotion, and external

rewards that signify status and respect for the profession are generally

equally important in recruitment and retention. In the sample of teachers

studied, private-school teachers who could afford the lower salaries that

their schools usually paid were more satisfied with their workplaces than

were their public school peers.

Traditional supervisory practices do little to improve the overall quality of

teaching. Most administrators do not have either sufficient pedagogical

and subject area skills to help proficient teachers or time for extended and

multiple observations. Teachers profit from training others, collaborative

activities, and peer reviews involving extended observations, student

evaluations, and self-assessment practices.

Virtually all teachers from both public and private sectors agreed that

supervision for improved performance should be separate from evaluation

decisions that determine salary or job status. None expressed confidence

that merit pay could be implemented fairly.

Comments Given the relatively small size of this sample and the diversity that it

included, the findings should be considered suggestive.

Page 27 GAO/PEMD-95-28 Schools and Workplaces
Appendix I 

Schools: Annotated Bibliography

Human Resources

Lieberman Ann Lieberman. “Practices That Support Teacher Development.” Phi Delta

Kappan, 76:8 (1995), 591-95.

Practice Teacher development.

Setting Selected successful schools across the nation.

Source of Information A review of the history of teacher development activities and related

research and an examination of practices at successful schools.

Findings and Conclusions If school improvement plans are to work, new and expanded teacher

development models are needed to teach knowledge and strategies. Unlike

conventional teacher development approaches that provide short-term

instruction outside the school and classroom, newer approaches provide

long-term, continuous learning within them. Newer approaches also

actively involve the teacher, unlike conventional ones.

Some of the new approaches expand the teacher’s role (as teacher-leaders,

peer coaches, or teacher-researchers). Others establish professional

groups within the school to solve problems, make decisions, or learn

about particular learning strategies. External networks, partnerships, and

coalitions can also effectively provide access to new ideas and support for

implementing them.

Comments The success of a new approach depends on the commitment of principals

and teachers to improving instruction through collegial and participatory

action and on their ability to evaluate its effect on student learning. How

the new approaches affect student achievement has not been documented.

Page 28 GAO/PEMD-95-28 Schools and Workplaces
Appendix I 

Schools: Annotated Bibliography

Human Resources

Little Judith Warren Little. “District Policy Choices and Teachers’ Professional

Development Opportunities.” Educational Evaluation and Policy Analysis,

11:2 (1989), 165-79.

Practice District programs for teachers’ professional development.

Source of Information Survey of practice in 30 California school districts, sampled according to

size. The study collected descriptions of local practice, data on more than

800 discrete staff development activities, and information from teachers

and administrators in these and other districts.

Findings and Conclusions Districts control much of the money that is available for staff

development, although teachers have some say about what is offered. The

bulk of the funds pay for central-office specialists or external consultants

(who typically have little connection to the district) to conduct workshops

or other activities. The emphasis of professional development programs

tends to reflect current state or district priorities and to vary from year to

year.

The range of activity is determined largely by the marketplace of

presenters and programs—by what is available and packaged to fit the

limited time available. Most activities are generic and are not tailored to a

specific curricular area or degree of teacher experience. Most are

relatively short term and entail little or no follow-up after the teacher has

returned to the classroom.

District evaluation of staff development is generally confined to soliciting

teachers’ opinions at the close of each activity. The practices summarized

above are not promising. Research has established that without follow-up

“coaching” or other peer support, teachers are unlikely to incorporate new

ideas (especially generic ones) into their teaching. Staff development

unconnected to broader efforts to improve the school, the curriculum, or

an individual’s career development plan is similarly unlikely to yield much

benefit.

Comments The study is limited by its absence of outcome data. The author’s judgment

is based on research on effective practices in staff development.

Page 29 GAO/PEMD-95-28 Schools and Workplaces
Appendix I 

Schools: Annotated Bibliography

Human Resources

Office of Technology

Assessment

Office of Technology Assessment. Teachers and Technology: Making the

Connection. Washington, D.C.: 1995

Practice In-service training: training teachers to use technology to enhance

learning.

Setting Public and private elementary and secondary schools throughout the

nation.

Source of Information Broad-based literature review; site visits to schools at all grade levels;

interview with teachers, administrators, and researchers; case studies of

exemplary training programs; survey of faculty and recent graduates of

colleges of education; review of past and present federal programs; and

analysis of trend data from several studies on the use of new technologies.

Findings and Conclusions By 1995, schools will have 5.8 million computers for use for instruction,

about one for every nine students. However, most teachers have not been

adequately trained to use computer technology in teaching. Most funds for

technology are spent on hardware and software, with districts devoting at

best 15 percent of funds to teacher training. Experts recommend doubling

this.

The current approach to teacher technology training is typically a short

in-service course for large numbers of teachers on one topic. This is

particularly inappropriate for technology training, which requires

hands-on work with software and hardware, curriculum specific

applications, and follow-up support.

Lack of teacher time, inadequate training options, rigid school scheduling,

and attitudinal barriers limit teachers’ ability to learn to use technology.

They have little compensated staff development time, and multiple

demands compete for it.

Most training emphasizes the mechanics of operating equipment with

scant information about how to integrate technology into classroom

instruction, choose software, or organize classes for maximum

technological benefit. On-site support to help teachers use technology is

rare. Only 6 percent of elementary and 3 percent of high schools have

full-time computer coordinators.

Page 30 GAO/PEMD-95-28 Schools and Workplaces
Appendix I 

Schools: Annotated Bibliography

Human Resources

Pauly Edward Pauly. The Classroom Crucible: What Really Works, What Doesn’t,

and Why. New York: HarperCollins, 1991.

Practice Traditional school district hiring.

Setting Mainstream elementary and secondary schools.

Source of Information Site visits and interviews with principals, teachers, students, and parents

in schools in California, Connecticut, Massachusetts, New York,

Pennsylvania, and Virginia. Review of relevant research.

Findings and Conclusions For several reasons, many schools hire and continue to employ teachers

who are not effective:

• the need to have a teacher for each classroom and an unwillingness to

raise taxes to pay the salaries that will attract a better teaching force,

• an unwillingness to spend money on a personnel system that can evaluate

applicants critically (by targeting the recruitment of good teachers or by

observing applicants in student teaching or current classroom situations),

• a reliance on centralized hiring, and

• an unwillingness to spend the time required to document a teacher’s

incompetent performance as grounds for dismissal.

Management strategies for retraining, reassignment, early retirement,

career change subsidies, and termination can reduce the number of

unsuccessful teachers but not unless traditional recruiting, hiring,

placement, and evaluation strategies are changed. In the schools studied,

assigning new teachers to classrooms with large numbers of students who

were difficult to teach and assigning more-experienced teachers to

less-demanding classrooms obstructed teaching success.

Comments Findings are based primarily on observational data but are supported by

external educational research evidence.

Page 31 GAO/PEMD-95-28 Schools and Workplaces
Appendix I 

Schools: Annotated Bibliography

Human Resources

Sparks and Loucks-Horsley Dennis Sparks and Susan Loucks-Horsley. “Models of Staff Development.”

In W. Robert Houston, M. Haberman, and J. Sikula (eds.). Handbook of

Research on Teacher Education. New York: MacMillan Publishing

Company, 1990.

Practice Staff development for teachers.

Source of Information Research studies and reviews of research on effective teacher

development practices.

Findings and Conclusions Research since 1980 has demonstrated that staff development can enhance

teacher performance and improve student learning. Effective approaches

include

• Learning activities undertaken or initiated by teachers to meet individually

defined needs or goals or to explore issues of common interest.

• Systematic observation and assessment of classroom teaching, followed

by feedback and discussion—an approach sometimes called peer coaching

or clinical supervision, which is especially powerful when used as a

follow-up to training in effective instructional practices.

• Participation in curriculum development or school improvement

processes, in which teachers design a strategy for solving a school

problem or attaining a goal.

• Training to improve teachers’ knowledge, thinking, and classroom skills. A

complete training process includes presentation and demonstration of the

new skills, trainee practice of the skills under simulated conditions, and

follow-up coaching and feedback in the trainee’s own classroom.

These approaches work best in schools where staff share common goals

and high expectations and administrators promote collegiality, place a

high priority on continuous improvement, and provide the resources

needed to support it.

Comments The training approach is well supported by data, including student

achievement data. The other approaches are supported by theory, teacher

perceptions, and case examples, but studies of them have rarely included

data on student outcomes.

Page 32 GAO/PEMD-95-28 Schools and Workplaces
Appendix I 

Schools: Annotated Bibliography

Curriculum and

Instruction

Alexander et al. Karl Alexander et al. On the Success of Failure. Cambridge, Eng.:

Cambridge University Press, 1994.

Practice Retention in the primary grades.

Setting Elementary schools in the Baltimore City Public School system.

Source of Information Longitudinal study that tracks a sample of 775 grade 1 students for 8 years.

Findings and Conclusions Retention is applied to (1) students who enter school far behind and make

little progress in first grade and (2) students whose performance is

adequate initially but declines sharply by the end of some later grade.

Retention boosts achievement and self-confidence during the repeated

year. Retained students subsequently lose ground but not as rapidly as

before. Over the long run, the boost from retention appears to have little

benefit for students in the first group or those who are put into special

education. However, some students in the second group get back on track

at modest levels of achievement.

Students who are retained are likely to be placed in the “low” group

throughout their primary years and in remedial courses in middle school.

The ill effects traditionally attributed to retention may instead reflect the

deficits that preceded retention plus the cumulative effects of low-track

placement. Being overage for a grade appears to contribute to problems

adjusting to middle school.

In view of retention’s short-term benefits and its ability to stem

performance decline among those who start out only modestly behind, it

must be considered a qualified success and a viable option. (Decisions

concerning retention should be made case by case.) The real need,

however, is to figure out how to avoid failure in the first place.

Comments Retention may have different effects in districts where very few students

are retained. The study is unusual in that it documents the academic

deficits that led to retention and students’ educational placement

subsequent to retention. It followed students only to middle school and

thus does not examine the effects of retention on dropping out.

Page 33 GAO/PEMD-95-28 Schools and Workplaces
Appendix I 

Schools: Annotated Bibliography

Curriculum and

Instruction

Becker Henry Jay Becker. “Computer-based Integrated Learning Systems in

Elementary and Middle Grades: A Critical Review and Synthesis of

Evaluation Reports.” Journal of Educational Computing Research, 8:1

(1992), 1-41.

Practice Integrated learning systems as the central component of the instructional

program.

Setting Elementary and middle schools.

Source of Information Reanalysis of the results of 30 evaluations of integrated learning systems.

Findings and Conclusions A growing number of districts have invested in integrated learning systems

technology (comprehensive software offering extensive instructional

activities and covering a range of subjects and grades) to teach core

academic skills. It has had a moderately positive effect on student

achievement. The clearest research findings are that computer-based

activities, in general, are motivational and increase engagement because

students enjoy working with them.

Evaluation reports provide insufficient information to compare the effects

of integrated learning systems on achievement with those of other

computerized technologies or traditional approaches such as reducing

class size and increasing teacher development activities. Thus, evidence is

not available to help districts make purchasing decisions between

competing products. More research is needed to understand how

differences in the organization of instruction in classes that use such

products affect student outcomes.

Comments As the author cautions, the evaluations of integrated learning systems that

he reviewed contained methodological weaknesses that limit the ability to

generalize from the findings.

Page 34 GAO/PEMD-95-28 Schools and Workplaces
Appendix I 

Schools: Annotated Bibliography

Curriculum and

Instruction

Funkhouser et al. Janie Funkhouser et al. “An Idea Book on Extending Learning Time for

Disadvantaged Students: Deciding on a Strategy.” U.S. Department of

Education, Washington, D.C., 1995.

Setting Fourteen programs to extend learning time for disadvantaged students. All

programs extended instructional time beyond the time required in the

school day, week, or year. All demonstrated anecdotal evidence of success

or were reputed to be successful.

Source of Information Volume 1 presents analyses of strategies used by 14 programs that experts

had identified. Volume 2 presents a profile of each program, including

evidence of its success.

Findings and Conclusions Promising extended-time programs link the added time to the regular

school academic experience by giving students the opportunity to build on

the skills and knowledge that they have acquired in the regular school

program. This linkage involves communication and coordination between

regular and extended program staff to ensure that the added instruction is

relevant to the regular curriculum and academically appropriate for the

students.

Promising strategies for extended-time programs include setting goals for

students and establishing systems for monitoring student progress,

planning enrichment activities that involve hands-on experience or that

involve themes of interest to the students, and providing an appropriately

challenging curriculum.

Most extended-time programs are developed by schools or districts. Some

are developed by partnerships between schools and outside agencies.

Programs may be staffed by paid teachers or rely on volunteers.

Comments Data on student outcomes were specific to each site. Therefore, it was not

possible to compare the relative effectiveness of different extended-time

program models.

Page 35 GAO/PEMD-95-28 Schools and Workplaces
Appendix I 

Schools: Annotated Bibliography

Curriculum and

Instruction

Gutierrez and Slavin Roberto Gutierrez and Robert E. Slavin. “Achievement Effects of the

Nongraded Elementary School: A Best Evidence Synthesis.” Review of

Educational Research, 62:4 (1992), 333-76.

Practice Nongraded programs.

Setting Mainstream kindergarten to grade 3 and kindergarten to grade 4 programs.

Source of Information A synthesis of research findings that compares the achievement, as

measured by standardized academic tests, of students in two types of

nongraded groupings with that of students in traditional classroom

groupings.

Findings and Conclusions Students in comprehensive nongraded programs (those that use

nongraded groups for more than one subject) make greater academic

gains than students in traditional classroom groupings.

Research on less-comprehensive nongraded programs (those that

incorporate individual instruction such as one-to-one tutoring,

programmed learning, and learning activities) demonstrates similar gains

for students in nongraded and graded organizations, indicating that as

nongraded programs become more complicated, they lose their advantage.

Comments The nongraded programs employed a continuous process curriculum that

organized skills hierarchically and spanned the grades they included. It

might not be possible to generalize the findings to nongraded programs

that do not have continuous process curricula.

Page 36 GAO/PEMD-95-28 Schools and Workplaces
Appendix I 

Schools: Annotated Bibliography

Curriculum and

Instruction

Knapp et al. Michael S. Knapp et al. Academic Challenge for the Children of Poverty.

Washington, D.C.: U.S. Department of Education, 1992.

Setting Elementary schools in which the student population overall varied from

moderate to high levels of poverty.

Source of Information Qualitative and quantitative data from 140 classrooms in 15 districts:

student backgrounds, scores from standardized tests and other

assessments, classroom observations, teacher interviews and survey

results, and examination of instructional material.

Findings and Conclusions Compared to traditional practices, alternative instruction that emphasizes

meaning and understanding is more effective in teaching advanced skills

and engaging students in learning and at least as effective in teaching basic

skills. Alternative instruction methods are at least as effective for

low-performing students as for high-performers.

In mathematics, students exposed to alternative instruction perform

substantially better on measures of computation ability than students

taught only arithmetic skills. In reading and writing, alternative instruction

produces positive effects in all skill areas but one.

Supplemental programs, including Chapter 1 pullout, English as a second

language, and compensatory education services, have mixed effects on

achievement. Results indicate that they should give less emphasis to basic

skills and more to more-advanced skills.

Comments The findings are limited to the acquisition and retention of learning over a

12-month period in only three states.

Page 37 GAO/PEMD-95-28 Schools and Workplaces
Appendix I 

Schools: Annotated Bibliography

Curriculum and

Instruction

Legters, McDill, and

McPartland

Nettie Legters, Edward McDill, and James McPartland. “Rising to the

Challenge: Emerging Strategies for Educating Students at Risk.” In Alesia

Montgomery and Robert Rossi (eds.). Educational Reforms and Students

at Risk: A Review of the Current State of the Art. Washington, D.C.: U.S.

Government Printing Office, 1994.

Practice Retention and grouping of low achievers.

Setting Elementary and secondary schools enrolling students at risk.

Source of Information Review of research on traditional approaches to compensatory education.

Findings and Conclusions Schools have traditionally responded to low achievers by grouping them

together for instruction and by retaining them in grade. They can benefit

thus for subjects such as reading in the elementary grades, but the

whole-day grouping and high school tracking that are currently practiced

appear to do them more harm than good. Tracking can be limited in a

number of ways, such as by grouping students only for specific courses.

Research has not determined whether such approaches produce a more

positive learning climate for students at risk. However, research has

identified methods for making heterogeneous classrooms work well.

Retention as generally practiced also has many negative consequences,

including dropping out. Retained students who receive extra help make

progress but not as much as similar students who receive help and are

promoted. Retention appears to be useful in the very early grades and at

key transition points (such as high school entry) but only if it is

accompanied by high-quality special programs.

Responses to low achievement include special education and the Chapter

1 program. Special education for children who have mild academic

handicaps offers them little benefit beyond what the regular classroom can

provide. The Chapter 1 program has produced modest gains, but they tend

to fade.

Comments The authors do not discuss social promotion or the effects of tracking and

grouping on students other than low-achievers.

Page 38 GAO/PEMD-95-28 Schools and Workplaces
Appendix I 

Schools: Annotated Bibliography

Curriculum and

Instruction

Meisels Samuel J. Meisels. “Doing Harm by Doing Good: Iatrogenic Effects of Early

Childhood Enrollment and Promotional Policies.” Early Childhood

Research Quarterly, 7 (1992), 155-74.

Practice Four entrance and promotion practices: raising the age to enter school,

retaining children in grade, adding extra-year and transition programs, and

parents’ holding children out from kindergarten.

Setting Early childhood and elementary schools.

Source of Information Synthesis of research literature; analysis of data from the National

Education Longitudinal Study of 1988.

Findings and Conclusions Today’s students are often older when they enter and progress through

school than their counterparts were a decade ago. In many districts, 10 to

15 percent attend 2-year kindergarten programs. About 19 percent of all

students are retained at least once in elementary school. Forty-four

percent of African American boys are retained at least once. In two

localities, as many as 18 to 20 percent of boys and 9 to 13 percent of girls

were held out of kindergarten by their parents. Economically advantaged

parents are more likely to hold children out than are disadvantaged

parents.

These practices have been harmful. Raising the entrance age does not

improve later school outcomes and is costly. Students who are over age

for their grade are more likely to drop out. Retention in one grade appears

to increase this risk by 40 percent; in two grades, by 90 percent. The

negative effects of 2-year kindergarten programs are similar to those of

retention. Holding students out has raised the age span in kindergarten

from 12 to 24 months, increasing the pressure to escalate the curriculum

to meet the needs of older, more advantaged, and maturationally different

students within a classroom.

Comments This study looks at the effects of retention on students who are over age

for their grade placement. Effects on other students are not considered.

Page 39 GAO/PEMD-95-28 Schools and Workplaces
Appendix I 

Schools: Annotated Bibliography

Curriculum and

Instruction

Montgomery and Rossi Alesia Montgomery and Robert Rossi (eds.). Educational Reforms and

Students at Risk: A Review of the Current State of the Art. Washington,

D.C.: U.S. Government Printing Office, 1994.

Practice Instructional and organizational strategies for educating students at risk of

making inadequate educational progress.

Setting Elementary and secondary schools enrolling at-risk students.

Source of Information Review of research on student background and school environment

factors that place children at risk and strategies for educating such

children.

Findings and Conclusions The climate and typical instructional practices in schools attended by

at-risk students offer them few incentives or pressures to engage in school

activities and to invest the effort needed for academic success.

Promising instructional practices include (1) early prevention programs,

(2) education that is adapted to differences in cultural background and

learning styles, (3) curricula that focus on real-world learning and

integrate academic with vocational skills, (4) instruction that incorporates

adult volunteers as mentors or advocates, (5) cooperative learning and

other strategies that incorporate peer support, (6) adult or peer tutoring,

and (7) rewards for progress as well as for high levels of attainment.

Alternative assessments and technology have promise but are difficult to

implement broadly enough to provide consistent exposure to at-risk

students as they move from school to school.

Promising school organizational features include (1) smaller, special-focus

programs that offer close, sustained, positive contacts between teachers

and students; (2) decreased departmentalization and tracking, with

modifications in classroom procedure to help teachers deal with the

greater range of students, (3) closer connections with work or college, and

(4) greater involvement with parents and community organizations.

Comments The authors call these practices “promising” because there is at least

short-term or case study evidence of their success. Data limitation and

cautions are noted for each practice.

Page 40 GAO/PEMD-95-28 Schools and Workplaces
Appendix I 

Schools: Annotated Bibliography

Curriculum and

Instruction

Natriello, McDill, and Pallas Gary Natriello, Edward L. McDill, and Aaron L. Pallas. Schooling

Disadvantaged Children: Racing Against Catastrophe. New York: Teachers

College Press, 1990.

Practice Elementary school compensatory education programs of several types.

Secondary school programs aimed at improving academic success,

motivating students to attend and exert effort, and providing supportive

conditions outside school.

Setting Schools enrolling educationally disadvantaged students.

Source of Information Review of research on educational practices and evaluation studies of

specific approaches and programs.

Findings and Conclusions Educationally disadvantaged students benefit from attending kindergarten

and, particularly, whole-day programs. Effective approaches in the

elementary grades offer academically focused, teacher-directed activities

that teach students a hierarchy of skills, evaluate their progress frequently,

and adjust instruction to their individual needs. Examples include the

DISTAR program for teaching mathematics, reading, and language;

cooperative learning approaches; structured peer-tutoring programs; and

some computer-assisted instruction programs. Many of the more powerful

approaches are aimed at all students, not just the lowest achievers, and

hence have fallen outside the scope of traditional Chapter 1 programs.

Effective high school programs provide disadvantaged students with

(1) opportunities for academic success through remedial assistance,

rewards for growth, and classroom exercises that draw on a variety of

abilities; (2) positive social relationships with staff and peers and a sense

of membership in and ownership of the school; (3) a sense of the

relevance of school to their future lives; and (4) strategies to mitigate the

unfavorable effects of external conditions on their participation and

performance in school. The first of these is critically important, given that

lack of academic success is one of the strongest predictors of dropping

out.

Comments Reviews the strengths and limitations of Chapter 1, Upward Bound, and

Job Corps and many other specific approaches.

Page 41 GAO/PEMD-95-28 Schools and Workplaces
Appendix I 

Schools: Annotated Bibliography

Curriculum and

Instruction

Oakes Jeannie Oakes. Keeping Track: How Schools Structure Inequality. New

Haven, Conn.: Yale University Press, 1985.

Practice Low- or remedial-track placements.

Setting Secondary schools throughout the nation.

Source of Information Case studies of 25 geographically and demographically diverse junior high

and high schools and a synthesis of research literature.

Findings and Conclusions No group of students has been found to benefit consistently from being in

a homogeneous group. The prevalence of tracking is based on the

persistence of four assumptions. Three of these are false—that bright

students are held back in mixed groups, that students in low and average

groups do not develop better self-concepts and attitudes toward school

than those in heterogeneous groups, and that the placement procedures

used to separate students into groups fairly and accurately reflect their

past achievement and native ability. The fourth assumption may be

true—that it is easier for teachers to accommodate individual differences

and to teach and manage students in homogeneous groups (particularly if

they are bright)—but this benefit is not worth the negative educational and

social effects of tracking.

Students in different tracks have fundamentally different educational

experiences. Teachers of high-track classes demand critical thinking,

problem solving, and the evaluation and synthesis of knowledge. Teachers

of average classes emphasize a simplified version of the skills and

knowledge demanded in a high-track class. Teachers of low-track classes

require only simple memorization and comprehension.

Comments The 25 case study schools were not a nationally representative sample.

Page 42 GAO/PEMD-95-28 Schools and Workplaces
Appendix I 

Schools: Annotated Bibliography

Curriculum and

Instruction

Oakes and Guiton Jeannie Oakes and Gretchen Guiton. “Matchmaking: The Dynamics of

High School Tracking Decisions.” American Educational Research Journal,

32:1 (1995), 3-33.

Practice Low or remedial track placements.

Setting Secondary schools.

Source of Information Qualitative and quantitative case studies of three high schools, including

analyses of course offerings, enrollment practices, and background and

transcript data.

Findings and Conclusions High school tracking decisions result from the interaction of three factors:

(1) differentiated, hierarchical curriculum structure; (2) school structures

committed to both common schooling and accommodating differences

among students; and (3) political decisions within those structures that

influence the distribution of advantage.

Teachers and administrators viewed students’ abilities and motivation as

fixed. High-performing students were given access to large numbers of

high-quality courses and to a culture of high expectations. In contrast,

low-performing students were placed in courses that provided few

opportunities to learn and diminished their expectations. Therefore, the

schools offered programs that accommodated rather than altered abilities

and motivation.

High-achieving students received a disproportionate share of school

resources. More-qualified teachers were assigned college-track classes,

and counselors gave more time and consideration to these students.

College preparatory courses were less likely than others to be reduced

because of budgetary constraints. The college preparatory curriculum was

very stable because it was well defined by state college admission

requirements and logically sequenced. Other programs were less coherent

and subject to cutbacks.

Comments Basing the report in case studies of three high schools allowed an in-depth

analysis of a wide variety of source data. However, it limited the ability to

generalize from the findings.

Page 43 GAO/PEMD-95-28 Schools and Workplaces
Appendix I 

Schools: Annotated Bibliography

Curriculum and

Instruction

Office of Technology

Assessment

Office of Technology Assessment. Linking for Learning: A New Course for

Education. Washington, D.C.: 1989.

Practice Distance-learning technologies to improve the quality of student education

and teacher training.

Setting Selected elementary, secondary, and postsecondary schools.

Source of Information A comprehensive overview of state and district policies and practices, a

review of the literature, and site visits to districts implementing successful,

leading-edge practices.

Findings and Conclusions New distance-learning technologies provide advanced and specialized

courses to students and training to teachers, allow interaction between

distant learner communities, and bring experts and information into the

classroom. Telecommunications technologies, including fiber optics and

computer linkage systems, enable geographically dispersed learners and

teachers to ask questions, discuss ideas, and receive quick feedback.

For adult learners and advanced high school students, distance learning

appears to be as instructional as classroom learning. Its relative

effectiveness for all students is not known.

States and districts are embracing distance learning as cost-effective in

meeting the diverse needs of students who are academically talented,

culturally isolated, disabled, home and hospital bound, and in need of

specialized motivational, enrichment, or remedial activities.

Comments This study’s eclectic strategies provided broad coverage of a variety of

distance-learning issues and settings. More research is needed to compare

the effectiveness of different technologies on student outcomes.

Page 44 GAO/PEMD-95-28 Schools and Workplaces
Appendix I 

Schools: Annotated Bibliography

Curriculum and

Instruction

Slavin and Madden Robert E. Slavin and Nancy A. Madden. “What Works for Students at Risk:

A Research Synthesis.” Educational Leadership, 46:5 (1989), 4-13.

Practice Instructional practices to accelerate learning. In continuous-progress

programs, students proceed at their own pace through a series of defined

instructional objectives in small groups with similar skills. Cooperative

learning programs, in which students work together in small groups to

master material presented by the teacher and receive group rewards, have

been effective in accelerating achievement in basic skills.

Setting Preschool and elementary schools enrolling at-risk students.

Source of Information A synthesis of research findings that compares student reading and

mathematics achievement in traditional classes to achievement in classes

with instructional practices designed to prevent failure by providing extra

resources, such as more time on learning tasks.

Findings and Conclusions Continuous progress and cooperative learning in grade 1 prevention

programs—both of which provide intensive resources such as tutors and

small-group instruction—increase reading achievement more successfully

than conventional programs, including in-class and pullout remedial

models.

Prekindergarten programs have strong positive effects on the language

and learning ability of disadvantaged children immediately after the

preschool experience. These tend to diminish in each subsequent year.

Other long-term effects (such as graduation rates and delinquency) appear

to be positive but data are limited.

Comments The study’s findings on the effects of preschool do not control for the

quality of subsequent educational experiences.

Page 45 GAO/PEMD-95-28 Schools and Workplaces
Appendix I 

Schools: Annotated Bibliography

Curriculum and

Instruction

Stringfield et al. Sam Stringfield et al. Urban and Suburban/Rural Special Strategies for

Educating Disadvantaged Children. Washington, D.C.: U.S. Department of

Education, 1994.

Setting Twenty-five urban, suburban, and rural school districts. All were eligible

for or participating in the federal Chapter 1 program, and all had

implemented special strategies that held promise for educating

disadvantaged students.

Source of Information Observations of classroom behavior; surveys of principals, teachers,

parents, and students; interviews with school staff; and analysis of

standardized test scores.

Findings and Conclusions The special strategies fall into three categories: philosophical, schoolwide,

and adjunct. Prerequisites for success in all include a faculty desire for

change, well-managed settings, funds for additional resources, principals

with effective leadership skills, effective staff, and parental involvement.

Successful philosophical strategies are based on the ideas of the Coalition

of Essential Schools, the Paideia Program, and Comer’s School

Development Program. They present the greatest challenge to implement

and replicate because they involve changing curriculum, school

decisionmaking, and instructional methods.

Successful schoolwide strategies, funded as a Chapter 1 option, depend

more on the principal’s leadership skills than the others do. They typically

use Chapter 1 funds to reduce class size, eliminate pullout services, extend

school time, improve staff development, purchase materials, and

implement adjunct programs.

Adjunct strategies such as Success For All, Reading Recovery, extended

time, and peer tutoring programs are the least intrusive and the easiest to

implement and replicate because they do not change traditional structures

or curriculums.

Comments The authors discuss findings only in terms of a special strategy. They do

not consider the effects of school or neighborhood characteristics that are

not associated with the three strategies.

Page 46 GAO/PEMD-95-28 Schools and Workplaces
Appendix I 

Schools: Annotated Bibliography

Curriculum and

Instruction

Walberg Herbert J. Walberg. “Enhancing School Productivity: The Research Basis.”

In Pedros Reyes (ed.). Teachers and Their Workplaces. Newbury Park,

Calif.: Sage Publications, 1990.

Practice Numerous instructional methods (such as reinforcement, accelerated

learning, programmed instruction, and reduced class size) and supportive

practices (such as graded homework, class morale, and television

viewing).

Setting U.S. public and private elementary and secondary schools.

Source of Information Meta-analyses of several thousand investigations of factors related to

learning, reviews of productive factors in learning.

Findings and Conclusions Of the instructional practices studied, reward or reinforcement for correct

performance had the largest overall average effect on learning.

Accelerated learning programs and training in reading ranked second and

third. Other methods, including cooperative-team learning, personalized

instruction, and tutoring also had strong effects. Smaller class size and

programmed instruction had, on the average, very little effect.

Of the supportive practices studied, graded homework, high class morale,

and family involvement in learning had strong positive effects on learning.

Twelve hours or more of leisure-time television viewing a week had a

negative effect.

The author concluded that improving the amount and quality of

instruction, and enlisting the involvement of families, can vastly improve

academic learning.

Comments The methodology combines the outcomes of a vast number of studies

about a particular practice into a common base of information. The author

could not report many details of the original studies, including those

relating to research design, methods, and conditions, because of the large

number of studies analyzed. The author encourages individuals who want

a more complete and critical understanding of specific practices to read

the original research.

Page 47 GAO/PEMD-95-28 Schools and Workplaces
Appendix I 

Schools: Annotated Bibliography

Curriculum and

Instruction

Wheelock Anne Wheelock. Crossing the Tracks: How Untracking Can Save American

Schools. New York: The New Press, 1992.

Practice Multilevel instructional groupings.

Setting Mainstream urban and suburban elementary and secondary schools.

Source of Information Review of selected research and classroom observations and interviews at

sites with multilevel instructional groupings.

Findings and Conclusions Multilevel instructional groupings require abandoning tracks that sort

students by perceived ability. Multilevel instruction expands their

exposure to knowledge and enables them to prepare for highly skilled

occupations.

Students’ learning appropriate grade-level skills in groups of mixed ability

is contingent on compatible instruction practices: a meaningful,

multifaceted curriculum; team teaching opportunities; multiple and

flexible opportunities to offer students extra help; and greater professional

development in new instruction techniques.

Comments The schools that had moved from ability tracks to heterogeneous

groupings had changed core curriculum, instruction, and assessment

practices, changes that were supported by explicit state and district

policies allowing them the autonomy to change, protection from

resistance, and financing to purchase resources.

Page 48 GAO/PEMD-95-28 Schools and Workplaces
Appendix II 

Workplaces: Annotated Bibliography

Work Environment

Bower and Christensen Joseph L. Bower and Clayton M. Christensen. “Disruptive Technologies:

Catching the Wave.” Harvard Business Review, 73:1

(January-February 1995), 43-53.

Practice Failure of successful companies to plan for customers of the future. The

downside of staying close to the customer.

Source of Information Selected case studies.

Findings and Conclusions Most established companies are consistently at the head of their industries

in developing and commercializing new technologies. Keeping focused on

main customers can blind companies to important new technologies in

emerging markets. The technological changes that typically damage a

company are not usually radically new or different but typically have a

package of performance attributes not initially valued by customers. The

performance attributes that existing customers in established markets do

value improve so rapidly that the new technology can later invade them.

While disruptive technologies may perform worse along one or two

dimensions, they generally make possible the emergence of new markets.

Rapid innovations can then improve performance and even satisfy the

needs of customers in the established markets. Sony’s early transistor

radios are an example of a new technology that sacrificed sound fidelity

by offering new attributes—small size, light weight, and portability.

Because managers are evaluated on their ability to place the right bets, it is

not surprising that well-managed companies back projects in which the

market seems ensured. Established companies need to build a

disruptive-technology business in an independent section of the

organization. Risk is reduced and careers are safeguarded, but the

corporation consists of business units with finite life spans: technological

and market bases of any business will eventually disappear.

Comments Disruptive technologies are an important part of the business cycle and

the forerunner of leading-edge technology.

Page 49 GAO/PEMD-95-28 Schools and Workplaces
Appendix II 

Workplaces: Annotated Bibliography

Work Environment

Boyett and Conn Joseph H. Boyett and Henry P. Conn. Workplace 2000: The Revolution

Reshaping American Business. New York: Plume Books, 1992.

Practice General management changes.

Source of Information The authors, who are management consultants with a large international

firm, researched material from a variety of sources: the popular press,

academic research, case studies of the best practices in U.S. industry, and

the advice and assistance of various individuals at national productivity

and quality associations.

Findings and Conclusions Recent changes in management and compensation practices were not

planned and American management did not set out to create new

workplace conditions. The flattening of organizations, the greater sharing

of information with employees, the motivation and leadership practices,

the pay systems and team systems, all were a response to competitive

need. American businesses had to adapt to survive.

Workplace practices and changes were not implemented in a systematic or

coordinated manner, but many experiments were successful. Case studies

of companies document their problems and progress. Key changes the

companies made provide guidance for others.

The workplace of the future will be flatter, leaner, and more aggressive.

Continued advancement in technology will require fewer employees in

organizations that will offer fewer opportunities for advancement.

Employment security may be a concept of the past, and although large

companies will offer a limited guarantee of long-term employment, it will

not necessarily be in the same job or workplace. This instability and

enormous competitive demands on businesses will put extraordinary

pressure on employees for maximum performance.

Comments The authors’ hypotheses are supported by selected case studies of

companies. Predictions about the future workplace practices of successful

American companies are based on a wide variety of sources.

Page 50 GAO/PEMD-95-28 Schools and Workplaces
Appendix II 

Workplaces: Annotated Bibliography

Work Environment

Collins and Porras James C. Collins and Jerry I. Porras. Built to Last. New York:

HarperCollins, 1994.

Practice “Visionary” companies that do not have flawless records but are resilient

and have “woven themselves into the fabric of society.”

Source of Information A comparison of 18 companies, from their beginnings to the present, that

since 1926 have outperformed the general stock market by a factor of 15 to

a matched set of companies that had similar opportunities but did not

attain the same stature. The authors ask, “What makes the exceptional

companies different from other companies?”

Findings and Conclusions Much of what are today called “new” or “innovative” management

methods—employee ownership, empowerment, continuous improvement,

total quality management, common vision, shared values—are versions of

practices that date to the 1800s. Contrary to popular belief, visionary

companies do not require a great idea or charismatic leaders. In the

combined 1,700-year life span of the visionary companies, the authors

found only four incidents of going outside for a chief executive officer, and

those in only two companies.

Visionary companies have unchanging core ideologies. Their underlying

dynamic is “Preserve the core and stimulate progress.” They do not need

to bring top management in from the outside for change and fresh ideas.

No one aspect makes them work: it is the dynamic of all the small pieces

together.

Comments The historical approach, while suggestive, clearly cannot be generalized to

other companies.

Page 51 GAO/PEMD-95-28 Schools and Workplaces
Appendix II 

Workplaces: Annotated Bibliography

Work Environment

Gaskill, Van Auken, and

Manning

LuAnn Ricketts Gaskill, Howard E. Van Auken, and Ronald A. Manning. “A

Factor Analytic Study of the Perceived Causes of Small Business Failure.”

Journal of Small Business Management, 31:4 (October 1993), 18-31.

Practice Business failure.

Source of Information Retail industry in Iowa. The study examined perceived causes of

small-business failure between 1987 and 1991 in apparel and accessory

retailing, consolidating many previous research findings and identifying

common themes. The Iowa Department of Revenue and all chamber of

commerce directors in Iowa were asked to identify failed business owners.

A telephone survey confirmed the information, and a questionnaire was

mailed to each of the businesses that were identified as having

discontinued because of financial reasons. The survey yielded a 70-percent

response rate.

Findings and Conclusions Four major factors explain the principal reasons for small business failure:

inadequate managerial and planning functions, poor relations with vendors

and difficulties receiving merchandise, inability to compete in the

competitive environment, and premature business growth or

overexpansion.

Comments The authors recognize that the research should be duplicated with other

geographic areas of the country and other business sectors if results are to

be generalized.

Page 52 GAO/PEMD-95-28 Schools and Workplaces
Appendix II 

Workplaces: Annotated Bibliography

Work Environment

Hogarty Donna Brown Hogarty. “Beating the Odds: Avoid These Mistakes at All

Costs!” Management Review, 82:2 (February 1993), 16-21.

Practice Common pitfalls that threaten new businesses.

Source of Information Management Review’s informal survey of venture capitalists, business

owners, consultants and other experts elicited their nominations for

common small-business set-up errors. Selected case studies.

Findings and Conclusions Countless product launches and new divisions fail each year in corporate

businesses for the same reasons that cause the demise of small business.

To avoid becoming another business failure, entrepreneurs need to learn

from others. The 10 most common new-business mistakes are

• failing to do initial market research,

• failing to find a market niche,

• not developing a business plan,

• developing an inadequate bookkeeping system,

• running out of money,

• failing to keep investors informed,

• not setting quality standards,

• being quick to hire and slow to fire,

• failing to get business early, and

• not asking for help.

Comments Results are based on nominations of various groups. The number of

respondents and sampling strategy were not provided, so that validity,

reliability, and the ability to generalize from the findings cannot be

ascertained.

Page 53 GAO/PEMD-95-28 Schools and Workplaces
Appendix II 

Workplaces: Annotated Bibliography

Work Environment

Jacob Rahul Jacob. “Corporate Reputations.” Fortune, March 6, 1995, pp. 54-94.

Practice The attributes of a company’s management, products, and services that

respondents to Fortune’s 13th annual Corporate Reputations (Most

Admired) Survey deem most important.

Source of Information More than 10,000 senior executives, outside directors, and financial

analysts rating the 10 largest companies in their own industry, taken from

companies in the 1994 Fortune 500 Industrial and Fortune 500 Service

directories. They measured the contenders by eight attributes: quality of

management, quality of product or services, financial soundness, value as

a long-term investment, use of corporate assets, innovativeness,

community or environmental responsibility, and ability to attract, develop,

and keep talented people. Their scores were averaged to reach the total.

Findings and Conclusions The bond between the individual and the organization becomes stronger in

high-performing companies. Successful companies have mastered the art

of maintaining continuity while fostering perpetual renewal. A company’s

culture is the characteristic that sets apart the top-ranking “most admired.”

Like character in a person, a company’s culture guides and defines,

allowing it to change and grow while remaining true to its core self. Such

culture can be passed down successfully through generations of

management.

Comments The sampling strategy and response rate were not provided. Employees

were not sampled.

Page 54 GAO/PEMD-95-28 Schools and Workplaces
Appendix II 

Workplaces: Annotated Bibliography

Work Environment

Kotter and Heskett John P. Kotter and James L. Heskett. Corporate Culture and Performance.

New York: The Free Press, 1992.

Practice Low-performance corporate culture.

Source of Information A small subset of 207 firms that focused on the largest 9 or 10 firms in 22

different U.S. industries. The findings are based on the histories of 20 firms

in diverse industries and geographical locations whose circumstances

have led to the development of cultures that undermine their economic

performance.

Findings and Conclusions A consistent pattern of events helps shape an important part of the

cultures of low-performance firms, a pattern that differs in at least one or

two critical respects from that found in the histories of high-performing

firms. The low-performance firms usually began with some combination of

visionary leadership and luck. Each firm implemented good business

strategy from a strong position in some market or markets and had the

means of sustaining that position. However, sustained growth and

increasing success created internal challenges. In response, executives

hired, developed, and promoted skilled managers who were not

necessarily leaders. They might have understood systems and budgets but

not vision, strategies, and culture. Eventually, any collective sense of why

the firm had been successful in the first place was lost.

Unhealthy cultures have three general components. First, managers tend

to be arrogant. At one company, no one was ever encouraged to look

outside the firm for new ideas; managers acted as if they already had all

the answers. Second, managers tend not to value highly customers and

stockholders and employees. They act insularly, sometimes politically.

Third, leadership and other mechanisms of change are treated with

hostility. These firms already had very strong managerial orientations, a

perspective that values stability, and order. Executives at some companies

who demonstrated “too much” leadership were often not promoted.

Comments The sample was based on corporations that had been successful at one

time. The economic performance was compared to other corporations in

the sample. Retrospective analysis makes it difficult to generalize from the

results.

Page 55 GAO/PEMD-95-28 Schools and Workplaces
Appendix II 

Workplaces: Annotated Bibliography

Work Environment

Labich Kenneth Labich. “Why Companies Fail.” Fortune, November 14, 1994, pp.

52-68.

Practice Main reasons why companies fail.

Source of Information Expert opinion.

Findings and Conclusions Some 97,000 U.S. businesses failed in 1992. Others lost ground to

competitors or key pieces of businesses disappeared. Senior managers

must take the responsibility. To avert failure, they must avoid

• identity crisis. Without a mental model of the business, decisionmaking

becomes capricious and the company drifts.

• failure of vision. In recent years, one of the most common disasters

stemming from management shortsightedness is getting stuck with

yesterday’s technology. Trouble starts when people are afraid to talk to the

boss and ideas do not get to the top.

• the big squeeze. Many companies have fallen into debt by paying too much

for an acquisition or overreacting to the predations of corporate raiders.

• the glue sticks and sticks. Companies fail when they cannot abandon

strategies that no longer work.

• anybody out there? Many companies fail because they have lost touch with

their most important customers.

• enemies within. Top managers, whatever their public declarations, who

take a narrow view of their employees can be subject to strikes or

hostilities. Corporate leaders who pump up their own bonuses and perks

while telling the troops to tighten their belts provoke discord and unrest in

the workplace.

Comments Six very general topics are presented with selected case studies. The

consensus of experts and retrospective analysis lack solid methodological

underpinnings.

Page 56 GAO/PEMD-95-28 Schools and Workplaces
Appendix II 

Workplaces: Annotated Bibliography

Work Environment

Levering Robert Levering. A Great Place to Work: What Makes Some Employers So

Good (And Most So Bad). New York: Avon, 1988.

Practice Essential elements that contribute to a satisfying workplace (and

descriptions of patterns of undesirable workplaces).

Source of Information Interviews with lower-level employees, top officers, and founders of 20 of

the best from The 100 Best Companies to Work for in America (Levering’s

1984 book co-authored with Milton Moskowitz). Research on management

theory, industrial psychology and sociology, economic history, and social

and moral philosophy.

Findings and Conclusions Creating a good work environment and increasing profits are compatible

goals. Some personnel fads (cafeteria benefit plans, quality circles,

employee stock ownership or 401(k) savings plans) genuinely improve

employees’ income and benefits, but a piecemeal approach is not the

essence of a great workplace. Most important is the quality of the

relationship between the company and the employees. Employees from

divergent companies use the same terms (trust, pride, freedom, family,

being treated fairly, being allowed to make mistakes); they are willing to

work hard but emphasize their need for a sense of respect.

Theories of management have serious political implications within a

company; they are not abstractions. Particular policies and practices

cannot define a great workplace, and successful companies are not

necessarily good workplaces. On the whole, the better employers enjoy

more financial success than their competitors.

Comments Findings from selected case studies cannot readily be generalized.

Page 57 GAO/PEMD-95-28 Schools and Workplaces
Appendix II 

Workplaces: Annotated Bibliography

Work Environment

Levering and Moskowitz Robert Levering and Milton Moskowitz. The 100 Best Companies to Work

for in America. New York: Doubleday, 1993.

Practice Descriptions of exceptional workplaces.

Source of Information Material from the authors’ Everybody’s Business: A Field Guide to the 400

Leading Companies in America and nominees from magazines,

newspapers, and others’ recommendations. Some companies nominated

themselves, and some individuals responded to inquiries for companies

with reputations as good workplaces. The authors asked the firms for

written material that would show why they should be considered and, at

the 147 top candidates, interviewed nonsupervisory employees, a similar

number of first-line supervisors and middle managers, human resource

executives, and the chief executive officer or another top officer. Then the

top 100 companies were profiled and rated for pay and benefits,

opportunities, job security, pride in work and the company, openness and

fairness, and camaraderie and friendliness.

Findings and Conclusions The top 100 companies are all highly successful. In downsizing their

workforces, 18 still adhere to a no-layoff policy, and several keep to it even

when confronted with steep erosions in profits. Employees believe that

job security is important and can easily detect manipulators—those who

use practices common to good workplaces simply to enhance the bottom

line. This is especially true in companies that jump on the quality

bandwagon and institute employee-involvement programs without altering

managerial attitudes or workstyle.

Comments Four of the companies have won the Malcolm Baldrige National Quality

Award. The authors resist generalizing that good workplaces are superior

because they help a company succeed. They acknowledge that no

company is perfect for everyone, particularly companies that have their

own culture, and try to indicate what kinds of people do and do not fit into

each company. They focus on the treatment of people during companies’

rocky periods.

The sampling procedure was not systematic or representative, so it is

impossible to generalize. Moreover, how each firm was rated on the

dimensions was not clear.

Page 58 GAO/PEMD-95-28 Schools and Workplaces
Appendix II 

Workplaces: Annotated Bibliography

Work Environment

Levine David I. Levine. Reinventing the Workplace: How Business and Employees

Can Both Win. Washington, D.C.: The Brookings Institution, 1995.

Practice Employee involvement.

Source of Information Empirical evidence, case studies, and social science theory.

Findings and Conclusions With varying degrees of success, employee involvement works. However,

broad employee involvement in the U.S. workforce remains the exception.

Although it can raise productivity, satisfaction, and product quality, it may

not always do so.

Various forms of representative participation can improve company

performance when they are part of a package of participatory policies.

Alone, they may improve labor-management relations but may have little

effect on productivity. Measuring the relationship between participation

and performance is exceedingly difficult.

The main obstacle to employee involvement is that many managers prefer

to retain control and avoid change. Another major obstacle is the

importance managers place on short-term gains: investment in employee

involvement is difficult to monitor, and the market initially yields low

results. Managers are given little incentive to raise employee involvement

to the level that would maximize profits. Further, unions have typically

gained power by confronting management. Employee involvement would

require new styles of interaction, skills, and risks.

U.S. employers train workers less than their foreign competitors do.

Companies reason that training workers who quit leaves them with no

return on their investment. Workers are unwilling to pay for much training

because they are unsure of the value of their training to other employers if

they are laid off; in any case, few employees can afford to pay tuition or

take the low wages that the company pays during the training period.

Young workers, who have the greatest need to acquire training, are

particularly likely to have no liquid assets.

Comments The author acknowledges that much of the book derives from research

and articles previously undertaken with co-authors. Numerous research

studies are cited.

Page 59 GAO/PEMD-95-28 Schools and Workplaces
Appendix II 

Workplaces: Annotated Bibliography

Work Environment

Levine and Tyson David I. Levine and Laura D’Andrea Tyson. “Participation, Productivity,

and the Firm’s Environment.” In Alan S. Blinder (ed.). Paying for

Productivity: A Look at the Evidence. Washington, D.C.: The Brookings

Institution, 1990. Pp. 183-241.

Practice The relationship between three broad forms of employee participation and

a firm’s productivity, accounting for the influence of the firm’s external

environment and industrial relations system. Forms of participation were

characterized as consultative, such as quality circles; substantive, such as

work teams; and representative, including workers’ councils, joint

labor-management consultation committees, and representation on

company boards.

Source of Information An extensive literature review, including a detailed analysis of 43 studies

relating participation and productivity in the United States, Europe, and

Japan. All 43 studies included quantitative measures of productivity.

Findings and Conclusions For conventional firms, most studies found no significant positive effect

from consultative participation on productivity, perhaps because

programs such as quality circles do not last long. But substantive

participation, especially on shop-floor issues, is usually related to higher

productivity, and increases in participation are related to higher

productivity gains. Representative participation is related to higher

productivity when part of a broader set of participatory policies but when

not standing alone. For employee-owned companies, most studies found a

positive relationship between participation and productivity, but these

companies also tend to have other characteristics that could affect

productivity, making it hard to draw definitive conclusions. Successful

participation systems have four characteristics: profit sharing or gain

sharing, long-term employment relations, increased group cohesiveness,

and guaranteed individual rights for employees. Whether firms adopt such

measures may be related to product, labor, and capital market conditions.

Comments The most important findings are based on reviews of empirical studies, but

the authors did not provide many details of the grounds on which they

selected studies to review. No meta-analysis of the studies was

undertaken.

Page 60 GAO/PEMD-95-28 Schools and Workplaces
Appendix II 

Workplaces: Annotated Bibliography

Work Environment

MacDuffie John Paul MacDuffie. “Human Resource Bundles and Manufacturing

Performance: Organizational Logic and Flexible Production Systems in the

World Auto Industry.” Industrial and Labor Relations Review, 48:2 (1995),

197-221.

Practice Effects of “bundles” of workplace practices on labor productivity and

product quality in the automobile industry. The workplace practices were

human resource management (such as the extent to which compensation

is based on performance, the extent of status barriers between workers

and managers, and the level of training given to employees), work systems

(such as formal work teams and employee involvement), and production

buffers (such as large parts inventories). An index was constructed for

each dimension, another for all three dimensions combined.

Source of Information As part of the International Assembly Plant Study sponsored by the

Massachusetts Institute of Technology, a questionnaire was administered

in 1989 to 90 auto assembly plants worldwide, representing 24 producers

in 16 nations and accounting for about 60 percent of worldwide capacity.

Extensive follow-up procedures filled in missing data and clarified

responses where necessary. This analysis is based on 62 responses.

Findings and Conclusions The relationships between the three indexes and the combined index, and

productivity and quality measures were analyzed with a hierarchical

regression model, with controls for levels of automation, production scale,

complexity of model mix, parts complexity, and age of production design.

All three indexes, and the combined index, predicted productivity and

quality at acceptable levels of statistical significance, except that the use

of buffers was not related to quality. The results support the hypothesis

that plants bundling human resource practices into an integrated or

flexible production system and business strategy outperform traditional

mass production systems in both productivity and quality.

Comments It is not possible to generalize broadly from this study of one

manufacturing industry. It was cross-sectional, so it cannot show causes

for productivity and quality from human resource practices. But it gives

strong evidence of the correlation between such practices and

performance in the automobile industry.

Page 61 GAO/PEMD-95-28 Schools and Workplaces
Appendix II 

Workplaces: Annotated Bibliography

Work Environment

Meeks, Ramos, and Palmeri Fleming Meeks, Steven Ramos, and Christopher Palmeri. “How to Tell an

Eagle from an Icarus.” Forbes, November 13, 1989, pp. 213-17.

Practice Successful small businesses.

Source of Information Forbes’ list of Up and Comers—100 Companies in 1979.

Findings and Conclusions Many things can happen to a small company in 10 years. Inflation, high

interest rates, and new technologies eliminated thousands of small

businesses and once-successful companies whose management failed to

adapt.

Of the initial 100 companies, 20 are worth less than they were 10 years ago.

The long-term winners, the top 15 companies, are niche companies.

Overall, the odds of success seem to favor strong little niche companies

with proven records.

Among other factors, the analysis seems to indicate that small businesses

that can adapt rapidly and have found a market niche outperform others.

Comments The initial criteria for selecting the company were not provided. The

retrospective analysis severely limits the ability to generalize, and

projecting the findings to other companies in other times would be

perilous.

Page 62 GAO/PEMD-95-28 Schools and Workplaces
Appendix II 

Workplaces: Annotated Bibliography

Work Environment

Miller Danny Miller. “The Icarus Paradox: How Exceptional Companies Bring

About Their Own Downfall.” Business Horizons, 35:1

(January-February 1992), 24-35.

Practice When successful practices are no longer effective.

Source of Information Long-term study of the evolution of outstanding firms. Case studies

illustrate and provide examples.

Findings and Conclusions The paradox of Icarus—whose greatest asset led to his demise—applies to

many outstanding companies whose successes and accomplishments

seduce them into excesses that cause their downfall. Overconfidence,

carelessness, and complacency are pitfalls for strategy, leadership, culture,

and structure. Many companies simply extend and amplify the networks

they credit for their success.

The dangers to large successful organizations are likened to great

civilizations in Arnold Toynbee’s A Study of History that collapsed or

stagnated from internal rigidity, complacency, and oppression.

Some successful organizations have adopted potentially powerful methods

for avoiding these problems:

• building thematic, cohesive configurations,

• encouraging managers to focus on the direction of the company,

• scanning widely and monitoring performance assiduously, and

• temporarily uncoupling renewal activities from established operations.

One way for a large organization to “have it all” is to establish small,

independent units that can experiment and do new things outside the

configuration of existing operations.

Comments Earlier research on outstanding companies yielded four “trajectories of

decline.” The author studied the long-term evolution of outstanding firms

conforming to these established types by tracking them for “many years.”

His research methods are not provided.

Page 63 GAO/PEMD-95-28 Schools and Workplaces
Appendix II 

Workplaces: Annotated Bibliography

Work Environment

National Center on the

Educational Quality of the

Workforce

National Center on the Educational Quality of the Workforce, “First

Findings.” EQW National Employer Survey. The Wharton School, Institute

of Research on Higher Education, University of Pennsylvania,

Philadelphia, 1995.

Practice Employers’ attitudes, practices, and expectations.

Source of Information Firms were identified by drawing on the World Bank’s multinational

survey of firms and their employees. A telephone survey was administered

in 1994 to more than 4,000 establishments. Firms in the manufacturing

sector and those with more than 100 employees were oversampled. The

survey was administered by the Bureau of the Census, and the study was

supported under the Educational Research and Development Center and

administered by the Office of Educational Research and Improvement,

U.S. Department of Education.

Findings and Conclusions High-performance work systems remain the exception: a quarter of the

establishments reported using benchmarking programs, and only

37 percent have adopted a formal total quality management program. Few

workers engage in high-performance work practices: 12 percent of

nonmanagerial workers participate in self-managed teams, only 17 percent

in job rotation. However, 54 percent of nonmanagerial employees

participate in meetings on work-related problems.

Only 5 percent of the establishments indicated any reduction in the skill

requirements of their jobs, while 56 percent reported increasing their skill

requirements. Virtually all establishments provide formal or on-the-job

training. Over half increased their formal training over the last 3 years.

More than 80 percent of their workers are fully proficient in their current

jobs. Although years of schooling and applicants’ skills certificates are a

factor in employers’ hiring decisions, they pay little attention to school

performance. More important is an applicant’s attitude, communication

skills, and history of successful work experience.

Comments The sampling strategy was not provided. The rate of cooperation and the

titles of the persons responding to the survey were not provided.

Therefore, representativeness remains uncertain.

Page 64 GAO/PEMD-95-28 Schools and Workplaces
Appendix II 

Workplaces: Annotated Bibliography

Work Environment

National Center on the

Educational Quality of the

Workforce

National Center on the Educational Quality of the Workforce. “The Other

Shoe: Education’s Contribution to the Productivity of Establishments.” A

second round of findings from the EQW National Employer Survey. The

Wharton School and The Institute for Research on Higher Education,

University of Pennsylvania, Philadelphia, 1994.

Practice Hiring decisions.

Source of Information A 1994 Bureau of the Census telephone survey of more than 4,000

establishments drawing from the work of the World Bank. The study was

supported under the Educational Research and Development Center and

administered by the Office of Educational Research and Improvement,

U.S. Department of Education.

Findings and Conclusions Increasing workers’ education produces twice the gain in workplace

efficiency that increasing the value of tools and machinery does. A

10-percent increase in education yielded an 8.6-percent increase in

productivity; a similar increase in capital stock (tools, buildings, and

machinery) yielded only a 3.4-percent productivity increase. The effect

rises to 11 percent in the nonmanufacturing sector, which has the greatest

job growth. Education had the lowest effect on productivity in wholesale

and retail trade. Employees in certain sectors were less likely to receive

education, and the training they did get was not portable, effectively

condemning them to low-paying jobs.

Some employers are divorced from schools and lack confidence in their

ability to train students for the workplace. They frequently base hiring

criteria on attitude, communication skills, and previous work experience.

Employers who base their hiring decisions on academic performance or

who have links to schools through internships or training programs show

higher productivity and more innovative workplace practices.

Comments The sampling strategy was not provided. The rate of cooperation and the

titles of the persons responding to the survey were not provided.

Therefore, representativeness remains uncertain. The study’s sponsors

plan to repeat the survey in 3 years because of the rapid pace of change in

the economy and labor market.

Page 65 GAO/PEMD-95-28 Schools and Workplaces
Appendix II 

Workplaces: Annotated Bibliography

Work Environment

Pullinger David Pullinger. “The Anorexic Organization.” Personnel Management,

24:3 (March 1992), 4.

Practice Cost-cutting.

Source of Information Expert opinion.

Findings and Conclusions An analogy is drawn between the cost-reduction efforts of some

companies and the anorexic young adult. Faced with managing a

difficulty, the classic response is to cut costs. This may leave an

organization leaner, but it may not be fitter. Cost-cutting may actually

weaken the organization’s ability to generate revenue. The more that

cost-cutting affects the company’s development activities, the more the

company’s ability to sustain itself in the long term will be impaired; it may

not be strong enough to defer bankruptcy or acquisitions.

Before implementing drastic measures, the organization should identify its

problems and their causes. The next step is to identify actions appropriate

to the issues. Short-term goals need to be set and met. At the same time,

the organization must engage employees in building for the future because

while it recovers and focuses on internal matters, the world moves on.

Markets change: opportunities open and close. Human resource personnel

have a significant role to play in the recovery process.

Comments This article is not based on a systematic analysis of representative data.

Page 66 GAO/PEMD-95-28 Schools and Workplaces
Appendix II 

Workplaces: Annotated Bibliography

Work Environment

Smith and Oliver Jim Smith and Mark Oliver. “The Baldrige Boondoggle.” Machine Design,

64:16 (August 6, 1992), 25-29.

Practice Quality awards.

Source of Information Selected case studies.

Findings and Conclusions High quality is not free and does not ensure success. Wallace Company

won the 1990 Malcolm Baldrige National Quality Award. It increased

on-time deliveries from 75 percent to 92 percent and increased market

share more than 8 percent but spent more than $2 million annually in extra

overhead. Customers had said they wanted more timely deliveries, but

they were not willing to pay extra for them.

Other companies, too, have discovered that high quality costs a great deal

of money and that customers prefer slightly less-than-perfect service from

firms that charge less. The fiscal performance of other Baldrige winners

has been mixed, and companies that have not won have consistently

outsold winners they compete with.

Some argue that the Baldrige Award was never meant to reward financial

success. Some exemplary companies have decided that pursuing the prize

is not cost-effective. The reality is that no company has performed better

than average after winning the Baldrige and most have had serious earning

declines.

The authors conclude that the Baldrige has virtue on its side.

Unfortunately, it takes virtue and solid business fundamentals to survive in

the modern world. A little less virtue and a lot more pragmatic business

sense is what America needs.

Comments Systematic sampling of Baldrige winners was not done.

Page 67 GAO/PEMD-95-28 Schools and Workplaces
Appendix II 

Workplaces: Annotated Bibliography

Work Environment

U.S. General Accounting Office U.S. General Accounting Office. Organizational Culture: Techniques

Companies Use to Perpetuate or Change Beliefs and Values,

GAO/NSIAD-92-105. Washington, D.C.: 1992.

Practice Developing and maintaining organizational culture, defined as the

underlying assumptions, beliefs, values, attitudes, and expectations shared

by an organization’s members.

Source of Information Discussions with five academicians who have done research on

organizational culture and officials of nine large companies, three of which

were attempting to perpetuate, and six to change, their corporate cultures,

particularly in regard to inventory management. The report grew out of a

series of studies on managing defense inventories.

Findings and Conclusions Changing an organization’s culture is a long-term effort, taking at least 5 to

10 years to complete. Company officials reported that two key techniques

for promoting cultural change are commitment of top management in both

words and actions and training that promotes and develops skills related

to the desired values and beliefs. In addition to these, company officials

indicated that successful change requires a combination of other

techniques, including distributing a written statement of desired values

and beliefs to employees; creating a management style that reinforces

these values and beliefs; offering rewards, incentives, and promotions to

support behavior that reinforces these beliefs; holding company gatherings

to discuss them; developing an organizational structure that is compatible

with them; using systems, procedures, and processes to support

organizational values; and using stories, legends, myths, and slogans to

communicate values and beliefs. Experts noted that the same techniques

could be used to perpetuate an existing organizational culture.

Page 68 GAO/PEMD-95-28 Schools and Workplaces
Appendix II 

Workplaces: Annotated Bibliography

Work Environment

Wagner John A. Wagner III. “Participation’s Effects on Performance and

Satisfaction: A Reconsideration of Research Evidence.” Academy of

Management Review, 19:2 (1994), 312-30.

Practice The effects of various forms of employee participation on employee

performance and job satisfaction: employee ownership, participation in

work decisions, and consultative, short-term, informal, and representative

participation.

Source of Information A meta-analysis of 11 previous literature reviews on the associations

between job performance and satisfaction and one or more forms of

subordinates’ participation in processing information, making decisions,

or solving problems.

Findings and Conclusions Employee participation is associated with improved job performance and

job satisfaction among employees, even though the studies used different

review methods (narrative, simple quantitative, meta-analytic) and

examined different samples of research studies. But while the association

is statistically significant, it is small; thus, if the costs of initiating and

maintaining such participation programs are high, companies are not

likely to adopt them. No statistically significant differences appeared

between the types of participation in terms of either performance or

satisfaction.

Comments The study’s statistical method might underestimate the variation in types

of participation but, given the small effect sizes, this is not likely to present

problems.

Page 69 GAO/PEMD-95-28 Schools and Workplaces
Appendix II 

Workplaces: Annotated Bibliography

Human Resources

Arthur Jeffry B. Arthur. “Effects of Human Resource Systems on Manufacturing

Performance and Turnover.” Academy of Management Journal, 37:3

(1994), 670-87.

Practice Differences between commitment systems and control human-resource

systems on the performance of steelmaking minimills. Compared to

control human-resource systems, commitment systems involve greater

decentralization of management, fewer workers per supervisor, more

employee participation in problem solving and decisionmaking,

higher-skilled workers, higher levels of general training, more formal

grievance procedures, more social activities, higher wages, and a higher

percentage of compensation accounted for by benefits and a lower

percentage by bonus or incentive payments.

Source of Information A survey of human resource managers at 30 of the 54 existing U.S.

minimills. The data were used to construct 10 variables on different

aspects of the mills’ human resource systems. Cluster analysis yielded six

clusters, but to preserve degrees of freedom, these were collapsed into the

commitment and control categories by judgment. (Reliability was tested

by six student raters.) Respondents also provided data on two measures of

performance (labor efficiency and scrap rate) and on turnover.

Findings and Conclusions Regression analysis found that commitment human-resource systems are

related to higher performance than control systems, signified by fewer

hours of labor to produce a ton of steel and a lower ratio of scrap to

finished steel. This was interpreted to mean that commitment systems

avoid a tradeoff between production quality (scrap rate) and efficiency

(labor hours). Commitment systems also have lower turnover than control

systems. Finally, the type of human resource system affects the

relationship between performance and turnover such that, in commitment

systems, higher levels of turnover consistently harm performance,

whereas this is not true for control systems.

Comments This study focused on only one manufacturing industry, so it cannot be

broadly generalized. The small sample size limited the analysis, so that

broad categories of systems had to be constructed, based on the author’s

judgment; and while the categorization was supported by other raters, its

validity and reliability remain at issue.

Page 70 GAO/PEMD-95-28 Schools and Workplaces
Appendix II 

Workplaces: Annotated Bibliography

Human Resources

Bassi Laurie J. Bassi. Smart Workers, Smart Work. Washington, D.C.: The

Southport Institute for Policy Analysis, 1992.

Practice Workplace education and reorganization of work.

Source of Information Case studies of 72 small and medium-size companies (fewer than 500

employees). Mail and telephone survey of a random sample of members of

the National Association of Manufacturers and a random sample of firms

from lists of a professional mail house.

Findings and Conclusions A third to a half of nonmanufacturing firms and half to three quarters of

manufacturing firms have undergone at least some reorganization of work;

only a small fraction have undergone substantial change in the way work

is done. The most frequently cited type of work reorganization is giving

greater responsibility to workers.

Larger firms are more likely to have reorganized work and implemented

education programs. Few firms have a workplace education program.

Among those that do, most commonly taught are skills for identifying

problems and techniques for their solution and interpersonal skills, such

team building. Manufacturing firms are much more likely to teach

academic skills such as reading, writing, mathematics, and the GED

curriculum, and improvements in terms of profits are most pronounced

from teaching them.

Firms that have instituted programs are not strikingly different from those

that have not, but they do have compensation packages that are equal to

or better than those of others; they are also more likely to promote from

within and have somewhat less turnover. The most frequently reported

direct effects of workplace programs are improved communication and

employee morale and greater customer satisfaction, self-confidence, and

ability to solve problems.

Comments The sample was drawn from an organizational membership and is not

necessarily representative of the industry. The survey yielded a response

rate of 18 percent, severely limiting any attempt to generalize from the

findings.

Page 71 GAO/PEMD-95-28 Schools and Workplaces
Appendix II 

Workplaces: Annotated Bibliography

Human Resources

Conte and Svejnar Michael A. Conte and Jan Svejnar. “The Performance Effects of Employee

Ownership Plans.” In Alan S. Blinder (ed.). Paying for Productivity: A Look

at the Evidence. Washington, D.C.: The Brookings Institution, 1990. Pp.

143-81.

Practice The relationship between employee stock ownership plans (ESOPs) and

related ownership forms and corporate performance in the United States

and, to a limited degree, Europe.

Source of Information Six studies of U.S. ESOPs supplemented by a brief review of studies of other

forms of employee ownership. Data from the studies were not aggregated

or meta-analyzed.

Findings and Conclusions The evidence on the effects of ownership through ESOPs on corporate

performance (productivity and profitability) is contradictory: some studies

show a direct effect, but they often lack control variables, which generally

reduces the effect to a negligible level. However, the evidence is stronger

that employee participation in decisionmaking in employee-owned

companies is associated with better performance. What the literature does

not address is whether participation has the same effect in

nonemployee-owned companies and whether employee ownership itself

leads to more participation.

Comments The authors were aware of variations in the strength of the studies they

reviewed and appropriately tempered their conclusions. The overall

findings are suggestive but hardly definitive.

Page 72 GAO/PEMD-95-28 Schools and Workplaces
Appendix II 

Workplaces: Annotated Bibliography

Human Resources

Cooke William N. Cooke. “Employee Participation Programs, Group-Based

Incentives, and Company Performance: A Union-Nonunion Comparison.”

Industrial and Labor Relations Review, 47:4 (July 1994), 594-609.

Practice Employee participation programs, group-based incentives, and company

performance.

Source of Information Survey of 841 manufacturing firms in five Michigan counties in 1989-90.

About 27 percent of the respondents were unionized.

Findings and Conclusions Employee-participation programs and group-based incentives yield gains

in a company’s performance when measured as value added net of labor

cost per employee. On the average, the unionized companies have higher

value added per employee, pay much higher hourly wages, have lower

ratios of labor costs to total costs, are less likely to have profit- or

gain-sharing plans, are older and larger, and employ a substantially larger

proportion of skilled tradesworkers than the nonunion firms.

Unionized companies with work teams but no group-based pay achieve

the highest level of performance—estimated at 35-percent higher than

comparable, nonunion firms without teams or group-based pay. Unionized

firms provide the better place for evaluating the benefits of

employee-participation programs; nonunion firms generally provide the

better place for estimating group-based pay as a performance incentive.

Comments Data came only from Michigan, although responses were received from

70 percent or more of the firms in each county. However, the regression

analysis was based on about 34 percent of the responses because data

were missing on key variables. Much more research is required, since the

data have limitations and since this is the first study to examine the effect

on company performance of the interactions between union

representation and employee-participation, profit-sharing, and

gain-sharing programs.

Page 73 GAO/PEMD-95-28 Schools and Workplaces
Appendix II 

Workplaces: Annotated Bibliography

Human Resources

Csoka Louis Csoka. “Closing the Human Performance Gap.” Report 1065-94-RR.

The Conference Board, New York, 1994.

Practice High-performance practices.

Source of Information A survey of human resource executives of a representative sample of

medium to large manufacturing and service firms requesting information

on current business practices in developing and maintaining a

high-performance workforce: 166 U.S. and European companies

responded. In-depth interviews were conducted with executives of

companies selected for their emphasis on innovations in human

performance improvement systems.

Findings and Conclusions Less than 5 percent of the companies spent more than $5,000 on internal

classroom and on-the-job training for new employees in the first year.

They placed more emphasis on external programs, such as partnering with

technical schools and high schools.

Although education and training are crucial to gaining a critical edge,

meeting the educational needs of the workforce is only part of the answer.

A systematic view of performance leads to outstanding learning programs.

Human resource programs emphasize accountability for business results

by focusing on the business and human resource practices that address

performance. The respondents show significant agreement on factors that

affect high performance. However, these factors do not correspond to the

causes of deficient performance. Survey results reveal that frequently

interventions are single, quick-fix programs that do not address underlying

causes of performance problems.

Comments The respondents were self-selected and self-identified as requesting

information on current business practices for developing and maintaining

a high-performance workforce. The number surveyed and the response

rate were not provided for this group.

Page 74 GAO/PEMD-95-28 Schools and Workplaces
Appendix II 

Workplaces: Annotated Bibliography

Human Resources

Cutcher-Gershenfeld Joel Cutcher-Gershenfeld. “The Impact on Economic Performance of a

Transformation in Workplace Relations.” Industrial and Labor Relations

Review, 44:2 (January 1991), 241-60.

Practice Conflict and cooperation in labor-management relations.

Source of Information Longitudinal analysis of changing patterns in labor-management relations

across work groups in a large unionized manufacturing facility over a

3-year period. Data were collected from union and employer records and

interviews.

Findings and Conclusions The focus was on two core elements of labor-management relations: the

resolution of disputes and the pursuit of common concerns. Work areas

with “traditional” labor-management relations, rooted in adversarial

assumptions, have higher costs, more scrap, lower productivity, and a

lower return to direct labor hours worked than work areas with

“transformational” relations, characterized by increasing cooperation and

improved dispute resolution.

Comments This company and the union are uncommon in that they have a long

tradition of positive relationships and the company has traditionally

placed a high value on human resources. Results of this study clearly

cannot be generalized.

Page 75 GAO/PEMD-95-28 Schools and Workplaces
Appendix II 

Workplaces: Annotated Bibliography

Human Resources

Fitz-enz Jac Fitz-enz. “The Truth About ’Best Practice.’” Human Resource Planning,

16:3 (1993), 19-26.

Practice Best practices.

Source of Information Company-operating data from the annual SHRM/Saratoga Institute Human

Resource Effectiveness Report. Screenings and multiple interviews with

the top 25 percent (defined by objective performance data) of 600

reporting companies. Interviews were designed to verify that the firm had

a plan, had succeeded, and could point to objective evidence of that

success in terms of productivity, quality, sales, or improved customer

service.

Findings and Conclusions The top 25 percent of companies have nine “best practices” in common.

Combining and redefining factors the following year confirmed eight best

practices in human resource management: communications, continuous

improvement, culture consciousness, customer focus and partnering,

interdependence, risk taking, strategy and commitment, and value focus.

“Best practice” is not a set of discrete actions but, rather, a cohesive,

holistic approach to organizational management that is the antecedent to

and transcends the visible activity.

Comments Case study and business article authors who focus on “best practices”

frequently come to misleading conclusions because they isolate a few

elements that supposedly account for a firm’s success. Because of this

narrow view, this literature presents an incomplete and shallow

description of activities. The author of this article selected a sample of the

top 25 percent of companies based on their performance data. From them,

he identified fundamental factors that drive a business enterprise. The

article’s findings are presented as a general guide rather than a specific

action plan, and consequently they are subject to wide interpretation.

Page 76 GAO/PEMD-95-28 Schools and Workplaces
Appendix II 

Workplaces: Annotated Bibliography

Human Resources

Greer and Ireland Charles R. Greer and Timothy C. Ireland. “Organizational and Financial

Correlates of a ’Contrarian’ Human Resource Investment Strategy.”

Academy of Management Journal, 35:5 (1992), 956-84.

Practice In the “contrarian” human resource investment strategy, a company

selectively hires key employees during a major business downturn, helping

it avoid personnel shortages, link human resource planning with corporate

strategy, provide access to higher-quality applicants than during better

times, assist in meeting affirmative action goals, ensure a regular age

distribution among employees, provide developmental and career planning

opportunities when business demands are relatively low, and contribute to

long-term financial performance.

Source of Information A three-wave survey of 500 firms drawn randomly from the 1987 Standard

and Poor’s Register of Corporations, Directors, and Executives that met

three criteria: (1) noninclusion in an earlier survey, (2) 2,000 or more

employees, and (3) listing on the 1986 Center for Research in Security

Prices monthly tape file. Financial performance indicators came from the

center’s data. The first wave produced 106 responses, the follow-up only

45. The third wave checked technical issues on the first two.

Findings and Conclusions Regression models showed the relationships between the benefits and

risks of selective countercyclical hiring along with contextual variables

(total company employment, industry, and product demand during

downturns) and total hiring, hiring of managers and professionals, and

self-report of corporate countercyclical hiring. Countercylical hiring is

positively related to avoiding personnel shortages, development and

career planning, and financial performance. Age distribution was mixed:

positive for hiring managers but negative for professionals (possibly

reflecting their higher pay and more rapid skill obsolescence). Costs are

negatively associated with countercyclical hiring, which may become

more important to long-term success as key projected labor shortages

emerge in the 1990s.

Comments The sample size was small (not more than 45 in most cases), making it

hard to generalize from the results. Since there were no data on the actual

performance of the companies, whether countercyclical hiring actually

produced the benefits expected could not be tested.

Page 77 GAO/PEMD-95-28 Schools and Workplaces
Appendix II 

Workplaces: Annotated Bibliography

Human Resources

Holzer et al. Harry Holzer et al. “Are Training Subsidies for Firms Effective? The

Michigan Experience.” Industrial and Labor Relations Review, 46:4

(July 1993), 625-36.

Practice State-financed funding for a training grant program for manufacturing

firms in Michigan with 500 or fewer employees that were implementing

some new type of technology.

Source of Information Survey of the amounts of formal job training (measured by the numbers of

employees participating in training programs and hours spent per

employee). Questions were also asked on the quality of output for each

year, using a variety of measures that firms use in their own quality control

programs.

Findings and Conclusions Training grants increase training in the year they are received but not

beyond. The quality of output, as measured by the scrap rate, is affected

positively, and the effect on training and scrap rate is “not completely lost

in subsequent years.”

Comments The first to apply for grants were the first to receive them. In 1988-89, 498

firms applied, and 270 were awarded grants. Only 24 percent of the firms

that received a grant responded to the survey, so the findings may contain

unknown biases.

Page 78 GAO/PEMD-95-28 Schools and Workplaces
Appendix II 

Workplaces: Annotated Bibliography

Human Resources

Kruse Douglas L. Kruse. Profit Sharing: Does It Make a Difference? Kalamazoo,

Mich.: Upjohn Institute for Employment Research, 1993.

Practice The effects of employee profit-sharing plans on productivity and stability.

Source of Information A telephone survey of 500 public companies, half with profit sharing for

employees other than management and half without. An attempt was made

to match profit-sharers with their counterparts in the same industry.

Productivity data were matched with publicly available data from public

companies on their 1970-91 financial characteristics and performance.

Findings and Conclusions The adoption of profit sharing is statistically associated with significant

productivity increases, with no negative postadoption trend. However, a

substantial number of adopters experience no productivity increase.

Increases are largest for small companies, cash plans, and plans with high

average contributions. Profit sharing does not appear to vary greatly by

occupational status or firm size but does appear to be less common among

unionized employees.

Advantages for workers and unions appear to be the potential for fewer

layoffs under adverse conditions and for higher pay from better

performance. Gains to employment stability accrue to the entire economy

as workers’ purchasing power is maintained without unemployment

insurance or government assistance. Profit sharing potentially yields

greater productivity, less unemployment, and macroeconomic stability,

which are central to economic performance, security, and the standard of

living.

Comments Companies with publicly traded stock are listed in Standard and Poor’s

CompuStat database. There was no information about how the sample was

chosen or how the paired control was selected. While the overall findings

are suggestive, they are not definitive.

Page 79 GAO/PEMD-95-28 Schools and Workplaces
Appendix II 

Workplaces: Annotated Bibliography

Human Resources

Mitchell, Lewin, and Lawler Daniel J. B. Mitchell, David Lewin, and Edward E. Lawler III. “Alternative

Pay Systems, Firm Performance, and Productivity.” In Alan S. Blinder

(ed.). Paying for Productivity: A Look at the Evidence. Washington, D.C.:

The Brookings Institution, 1990.

Practice The relationships between alternative pay systems—profit sharing, gain

sharing, stock options, ESOPs, and incentive or bonus plans—and on the

economic performance and productivity of firms. The effects of incentive

plans on wages and profit-sharing plans on compensation levels.

Source of Information For the main analysis, 495 responses to a survey of 7,000 business units

about 1986-87 human resource policies and practices (from the Columbia

(University) Business Data Set) matched to 1983-86 financial performance

data (from COMPUSTAT). For the relationship between incentive plans

and hourly wages, Bureau of Labor Statistics 1979-86 wage surveys; for the

relationship between profit sharing and other fringe benefits, National

Industrial Conference Board surveys from the early 1950s.

Findings and Conclusions Regression analyses showed that an index of the five alternative pay

variables is associated positively with higher productivity among

production workers but not clerical workers. Profit sharing is associated

with higher productivity for both types of employees; ESOPs are associated

with higher productivity among production workers. Similar results were

not found for performance, measured as return on investment or return on

assets. When related to changes in the productivity and performance

measures, both profit sharing and ESOPs showed positive associations.

Firms with such plans are able to respond more quickly to economic

downturns than other firms. Incentive plans are associated with higher

hourly wages, and profit-sharing plans do not appear to substitute for

pension or other fringe benefits.

Comments The main analysis was based on a survey with only a 6.5-percent response

rate, making generalizations impossible. The authors used indexes for

many of their variables, raising questions about whether their analytical

techniques were appropriate. Some data were quite old, in one case dating

to the 1950s.

Page 80 GAO/PEMD-95-28 Schools and Workplaces
Appendix II 

Workplaces: Annotated Bibliography

Human Resources

Snell and Dean Scott A. Snell and James W. Dean, Jr. “Integrated Manufacturing and

Human Resource Management: A Human Capital Perspective.” Academy

of Management Journal, 35:3 (1992), 467-504.

Practice Four human resource management practices—selective staffing,

comprehensive training, developmental performance appraisal, and

equitable rewards—and how they are related to three components of

integrated manufacturing—advanced manufacturing technology,

just-in-time inventory controls, and total quality.

Source of Information Surveys of plants of 512 firms in the metalworking industry, drawn from

the Harris Pennsylvania Industrial Directory; 160 plant managers

responded. In addition, human resource managers and nonmanagerial

employees were surveyed, with up to 120 responding. Multiple-item scales

were used as measures of the human resources management practices and

manufacturing innovations. Controls for organization and industry

environment were based on data from the surveys, Standard and Poor’s

Directory of Corporate Affiliations, and the Department of Commerce’s

Survey of Manufactures.

Findings and Conclusions The authors tested the relationships between the three components of

integrated manufacturing, including all their possible combinations, and

personnel practices. Firms using integrated manufacturing—especially

advanced manufacturing technology and total quality—are more likely

than traditional firms to exert effort to develop operations and quality

workers. Just-in-time inventory controls, however, are sometimes

negatively related to personnel practices; they may actually limit employee

discretion, reducing the need for innovations in personnel practices. The

interaction of all three components shows consistently negative

relationships with human-resource practices for production control

workers. The authors suggest that this is because when all these processes

are in place, production control may become superfluous, requiring less

human-resource investment.

Comments The sample size was small and drawn from only one industry. The study

did not address whether firms actually perform better when they use the

human-resources management practices studied.

Page 81 GAO/PEMD-95-28 Schools and Workplaces
Appendix II 

Workplaces: Annotated Bibliography

Human Resources

U.S. General Accounting Office U.S. General Accounting Office. Employee Stock Ownership Plans: Little

Evidence of Effects on Corporate Performance, GAO/PEMD-88-1. Washington,

D.C.: 1987.

Practice Employee stock ownership plans.

Source of Information Within a broader study of ESOPs, GAO compared the performance of a

subsample of 111 firms sponsoring ESOPs with that of a comparison sample

of firms that did not sponsor ESOPs. In the comparison sample, each firm

matched an ESOP firm on measures of industry and size. For each firm in

the two samples, GAO obtained corporate tax returns covering 6 years,

including the 2 years before an ESOP was established, the year it was

formed, and 3 years afterward. Data from these returns were used to

construct measures of productivity and profitability. Other data on ESOPs

and sponsoring firms came from a survey of 1,100 ESOP firms, with a

response rate of 77 percent.

Findings and Conclusions Using analysis of covariance techniques, GAO found that firms did not have

higher rates of productivity growth or profitability after they adopted

ESOPs, compared to the matching non-ESOP firms. GAO suggested that the

lack of ESOP effects on corporate performance could reflect a number of

factors. First, the ownership benefits of ESOPs appeared to be too limited

and too delayed to motivate higher performance levels among employees.

Because ESOPs are organized under retirement law, participants did not

have access to the stock until separation from the sponsoring company.

Moreover, few firms passed through stock dividends to employees, and

voting rights were limited. Second, although ESOPs were designed to assist

firms in financing capital expansion, few firms actually used them for that

purpose, so the potential productivity and profitability gains from

investments were necessarily limited.

In a separate analysis of the ESOP firms only, GAO found that those that also

featured a high level of employee participation in decisionmaking had

considerably higher productivity improvements after starting an ESOP.

Since this finding was only associational, GAO could not determine whether

ESOP firms that gave employees higher levels of participation improved

performance or whether better-performing companies called upon their

employees to participate more than did other firms.

Page 82 GAO/PEMD-95-28 Schools and Workplaces
Appendix II 

Workplaces: Annotated Bibliography

Human Resources

U.S. General Accounting Office U.S. General Accounting Office. Workforce Reductions: Downsizing

Strategies Used in Selected Organizations, GAO/GGD-95-54. Washington, D.C.:

1995.

Practice Downsizing an organization’s workforce.

Source of Information Reviews of documents and interviews with corporate and government

officials in 25 organizations reputed in literature searches and discussions

with experts and consultants to have successfully met downsizing goals.

The organizations included 17 companies, 5 U.S. state governments, and 3

foreign national governments. Information was collected on planning for

workforce reductions, approaches used to carry them out, and the human

resource aspects of the downsizing efforts.

Findings and Conclusions The private companies reported that their downsizing decisions were

related to restructuring decisions designed to make work processes more

efficient or eliminate unnecessary work; only seldom were they designed

to reduce employment. In contrast, the states reported that downsizing

was generally undertaken to cut costs, regardless of work considerations.

Of the 25 organizations, 15 reported that it was important to plan how

reductions would be carried out in order to retain a viable workforce.

Organizations that said they did not plan their downsizing efforts properly

acknowledged that they cut needed employees, suffered skills imbalances,

and were often forced to rehire or replace separated employees.

Most of the organizations used monetary incentives to encourage “at risk”

employees to leave voluntarily if they could not be redeployed. Among

these incentives were early retirement without penalties, credit of

additional years of service in retirement benefit determinations, and

lump-sum severance payments of up to one year’s salary. Attrition and

hiring freezes were reported to be useful but not always effective in

achieving short-term workforce reductions. Involuntary separations were

usually a last resort; where possible, they were targeted to meet

restructuring goals. The organizations were also concerned to minimize

disruptions through frequent and open communications, along with

counseling, outplacement, and retraining programs to help employees.

Page 83 GAO/PEMD-95-28 Schools and Workplaces
Appendix II 

Workplaces: Annotated Bibliography

Human Resources

Weitzman and Kruse Martin L. Weitzman and Douglas L. Kruse. “Profit Sharing and

Productivity.” In Alan S. Blinder (ed.). Paying for Productivity: A Look at

the Evidence. Washington, D.C.: The Brookings Institution, 1990. Pp.

95-141.

Practice Profit-sharing plans (including gain sharing and cooperatives) and their

relationships with corporate economic performance in the United States

and Europe.

Source of Information Five surveys of the case study literature, 6 employee surveys, 15 employer

surveys, 6 statistical studies, and 16 econometric studies that were

conducted between the 1950s and 1980s.

Findings and Conclusions Profit-sharing plans are related to higher productivity across all types of

studies. The strongest evidence comes from the econometric studies.

Meta-analysis shows that 60 percent of the 226 reported coefficients

relating profit sharing to productivity are positive and statistically

significant; the median estimated effect of profit sharing on productivity is

4.4 percent. Virtually all the statistical studies also reported higher

productivity for firms with profit-sharing plans than those without. Both

employers and employees reported positive profit-sharing effects on

productivity and firm performance, although employees showed concern

over fluctuating income. The case studies, too, reported positive results.

The evidence, while not definitive, supports the theoretical argument that

profit sharing is linked to higher productivity.

Comments Many of the studies were apparently methodologically weak, but the

authors do not sufficiently discuss their methods for selecting studies to

permit judgments about the adequacy of the evidence they present. The

literature review was probably, as most such reviews are, affected by a

bias toward the publication of positive results in the literature.

Page 84 GAO/PEMD-95-28 Schools and Workplaces
Appendix II 

Workplaces: Annotated Bibliography

Human Resources

U.S. Department of Labor U.S. Department of Labor. What’s Working (and What’s Not): A Summary

of Research on the Economic Impacts of Employment and Training

Programs. Washington, D.C.: January 1995.

Practice Employment, training, and education programs.

Source of Information Comprehensive review of more than 100 nonpartisan social science

studies on the economic effects of employment, training, and educational

programs.

Findings and Conclusions At least some programs have been successful for every population

examined: disadvantaged youths and adults, single mothers, and displaced

workers. A large number of programs for disadvantaged youths have not

produced significant earnings gains; the authors suggest that it is

important to help them complete high school. Programs for disadvantaged

women have probably demonstrated the most success. However, given the

low base earnings of mothers on welfare, even substantially positive

programs have not greatly reduced their poverty rates. Even a year or two

of successful training for displaced high-tenure jobs does not often lead to

income gains large enough to restore predisplacement earnings. Training

can blunt the effect of displacement on income but does not eliminate it.

On the average, the gains from successful programs are moderate.

However, they represent real gains for society and for individuals: returns

to society of $1.40 or more per dollar invested in some programs for

disadvantaged adults, welfare-to-work programs, and job search

assistance for displaced workers. The study’s findings argue against a

“one-size-fits-all” model and for an attempt to make a wide variety of

training choices more accessible to disadvantaged persons, those who are

unemployed, and those who need to upgrade their skills.

Comments This review summarizes major quantitative studies. Experimental designs

using randomly assigned individuals are included in addition to

quasi-experimental studies with matched comparison groups. The authors

acknowledge the difficulties in generalizing from a demonstration project

to “real world” situations. The paucity of acceptable impact studies may

bias the findings.

Page 85 GAO/PEMD-95-28 Schools and Workplaces
Appendix II 

Workplaces: Annotated Bibliography

Management

Processes

American Management

Association

American Management Association. “1994 AMA Survey on Downsizing.”

Research reports. New York, 1994.

Practice Downsizing.

Source of Information The sample consists of 713 human resource managers in AMA companies

that are members of the American Management Association. The

respondent base is a sampling of the association’s corporate membership

of 7,000 organizations, which together employ one fourth of the American

workforce.

Findings and Conclusions Almost half (47.3 percent) of respondents reported job eliminations in the

12 months ending in June 1994. Overall, profits rose at only 51 percent of

the companies that downsized between 1989 and 1994 and actually

declined in 30 percent. Worker productivity increased in 34 percent and

decreased in 30 percent of the reporting companies. Employee morale

declined in 88 percent of companies where downsizing was ongoing.

Two thirds of the companies that eliminated jobs also added new

employees to their payrolls. Due to this concurrent job creation, the

workforce actually increased at 13 percent of companies reporting job

cuts and did not change at another 25 percent. Downsizing targeted

salaried workers in general and middle managers in particular. Economic,

organizational, and technological issues made this group especially

vulnerable. The results indicate that, for the first time since 1988-89,

downsizing was strategic rather than reactive and cuts are being made

selectively to fine-tune operations.

Downsizing tended to be more prevalent in large companies (over 10,000

employees), but the cuts were not as deep as in previous years. And these

companies are also more likely to create new jobs as they downsize.

Comments Downsizing was defined as any elimination of jobs during the survey

period and not necessarily a net reduction in workforce. The study was

limited to association membership, and the response rate was not

provided. The author notes that the survey does not depict activity within

the U.S. economy as a whole but reflects policies and practices at major

U.S. firms.

Page 86 GAO/PEMD-95-28 Schools and Workplaces
Appendix II 

Workplaces: Annotated Bibliography

Management

Processes

Arthur Jeffry B. Arthur. “The Link Between Business Strategy and Industrial

Relations Systems in American Minimills.” Industrial and Labor Relations

Review, 45:3 (1992), 488-506.

Practice How variation in workplace industrial relations systems is related to

differences in business strategy in steelmaking minimills. Considers a

number of workplace practices, including the decentralization of

management, number of workers per supervisor, employee participation

in problem solving and decisionmaking, workers’ skills, general training,

grievance procedures, wage levels, percentage of compensation accounted

for by benefits and bonus or incentive payments, and social activities.

Source of Information Thirty responses from a survey of personnel managers of the 54 existing

U.S. minimills. The data were used to construct 10 variables on different

aspects of the mills’ personnel systems. Cluster analysis yielded six

clusters, grouped broadly as commitment maximization and cost

reduction. Plant managers at the mills were also surveyed on business

strategy. Using cluster analysis on eight variables, the author identified

four strategies: low cost and three variants of differentiation (versatile,

specialized, and customized).

Findings and Conclusions Matrix and regression analyses showed that personnel system types are

significantly related to business strategy types. Thus, low-cost producers

are more likely to have cost-reducing industrial relations systems, while

companies with differentiation strategies are more likely to adopt

commitment maximizing systems. This relationship holds even when

controls for region, age of mill, company size, unemployment, and

unionization are included in the analysis. The type of industrial relations

system that a mill adopts is related to its business strategy.

Comments This study focused on only one manufacturing industry, so it cannot be

broadly generalized. The small sample size limited the analysis. The study

was cross-sectional, so it is unclear whether an industrial relations system

was adopted to comport with the business strategy or the strategy was

contingent on the industrial relations system, although the latter would

appear to be less likely (except possibly where unions were involved).

Page 87 GAO/PEMD-95-28 Schools and Workplaces
Appendix II 

Workplaces: Annotated Bibliography

Management

Processes

Benson Tracy E. Benson. “IQSSM: Quality Is Not What You Think It Is.” Industry

Week, October 5, 1992, pp. 22-34.

Practice Quality practices that result in greater profitability, productivity, and

perceived quality.

Source of Information Further findings from the International Quality Study survey. The

American Quality Foundation and Ernst and Young (Cleveland) jointly

designed the survey to examine management practices that affect various

performance measures in companies around the world. Results are based

on 584 participating companies in four countries. Data were based on

questionnaires.

Findings and Conclusions Total quality management is not one thing or one set of initiatives to all

people. It is a management system designed and installed entirely for the

unique challenges to a company. There is no prescription of practices that

is right for every company.

Total quality management is, however, a system of interrelated parts. Not

everything one does necessarily affects quality, profit, or growth directly,

but one action may affect something else, which may then affect those

measures.

Comments Sampling strategy and response rate are not provided, raising questions

about the ability to generalize from the findings.

Page 88 GAO/PEMD-95-28 Schools and Workplaces
Appendix II 

Workplaces: Annotated Bibliography

Management

Processes

Foster Thomas A. Foster. “Searching for the Best.” Distribution, March 1992, pp.

31-36.

Practice Benchmarking.

Source of Information Survey of the Fortune 1000 companies.

Findings and Conclusions Benchmarking involves tasks at three levels. Lowest is working-task

benchmarking, covering single logistic activities. Next is function-wide

benchmarking, looking at all logistic tasks at the same time. Highest is

management-process benchmarking, in which managers look jointly at

broad issues such as quality and employee reward systems. The potential

payback at this level is enormous.

Sixty-five percent of the survey respondents revealed that their companies

use benchmarking as a management tool “to gain competitive advantage.”

However, experts warn that companies cannot limit themselves to

benchmarking competitors, since it is unlikely that they will find practices

that will allow them to outdistance their competition.

Asked to name their leading benchmarking partners, 300 major companies

named 67 in six different functional areas; a relatively few leading-edge

companies received the majority of mentions in most categories.

Comments Information regarding the size of the sample or the response rate was not

provided.

Page 89 GAO/PEMD-95-28 Schools and Workplaces
Appendix II 

Workplaces: Annotated Bibliography

Management

Processes

Hodgetts Richard H. Hodgetts. “Quality Lessons from America’s Baldrige Winners.”

Business Horizons, 37:4 (July-August 1994), 74-79.

Practice Total quality management. The attributes of recent Malcolm Baldrige

Quality Award winners, both the well-known firms and the lesser-known.

Source of Information Site visits, telephone interviews, and secondary data.

Findings and Conclusions Eight characteristics are universal to the Baldrige winners.

1. Every winner has a vision of what “quality” means and uses this vision

to guide its quality plan.

2. All top management is actively involved in the total quality effort.

3. Winners focus on customer needs, most commonly by gathering and

analyzing data on them.

4. All companies determine their objectives and then formulate a plan of

action. Some create an office or designate a senior manager to spearhead

the development of necessary infrastructure.

5. They train employees to use Statistical Process Control tools to help

them in solving problems.

6. They empower employees with the authority to take control and make

decisions.

7. They recognize successful employees positively, with financial rewards,

days off, vacation trips, choice parking spots, and the like.

8. They make continuous improvement an ongoing challenge. Two of the

most common tools for this are benchmarking and reduction of defects.

Comments The companies were self-selected, so the results cannot be generalized to

other firms.

Page 90 GAO/PEMD-95-28 Schools and Workplaces
Appendix II 

Workplaces: Annotated Bibliography

Management

Processes

Mavrinac and Jones Sarah C. Mavrinac and Neil R. Jones. “The Financial and Non-Financial

Returns to Innovative Workplace Practices: A Critical Review.”

Preliminary report for the U.S. Department of Labor. Office of the

American Workforce, Boston, Massachusetts, September 15, 1994.

Practice Various practices.

Source of Information More than 100 academic papers that examined the relationships between

firms’ financial and nonfinancial outcomes and various employee-,

process-, and quality-related workplace practices. These papers employed

a variety of methods.

Findings and Conclusions A firm’s financial performance and intermediate workplace outcomes, like

product quality and customer satisfaction, generally increase with

investments in workplace practices. The most popular and evidently

successful is total quality management, which emphasizes the integration

of employee development and process management to ensure continuous

improvement, customer satisfaction, and structured learning.

Data from 632 companies revealed that higher levels of product quality

were significantly associated with higher levels of financial performance in

three of six industry groups. One study found that firms noted for “above

average implementation” of total quality management earned more than

15-percent greater returns over the ensuing 5 years.

Some data failed to find an association between workplace

practices—which vary with technology, management skill, competitive

conditions, and time—and financial outcomes.

Comments The authors discuss the methodological strengths and weaknesses of

selected studies. Whether they took into account the relative strengths of

the papers in reaching their conclusions is unclear, but the evidence

presented appears to be solid.

Page 91 GAO/PEMD-95-28 Schools and Workplaces
Appendix II 

Workplaces: Annotated Bibliography

Management

Processes

Osterman Paul Osterman. “How Common Is Workplace Transformation and Who

Adopts It?” Industrial and Labor Relations Review, 47:2 (1994), 173-88.

Practice Various flexible workplace practices—self-directed work teams, job

rotation, employee problem-solving groups (quality circles), and total

quality management.

Source of Information A survey of a representative, stratified sample of nonagricultural, for-profit

U.S. business establishments with 50 or more employees, directed to the

most senior persons in charge of the production of goods or services. The

response rate was 65.5 percent, yielding a usable sample of 694 cases.

Detailed information was obtained only for “core” employees, the largest

group of nonmanagerial production or service employees.

Findings and Conclusions Flexible workplace practices are more likely in establishments in which

management values support increasing the personal and social well-being

of employees; those selling in international markets (regardless of the

overall level of competition in the industry); those whose strategy

emphasizes quality, variety, and service over cost competition; those

requiring more highly skilled workers; those belonging to larger

organizations; and small establishments. However, the adoption of flexible

practices is not related to an establishment’s time horizon (long-term

versus short-term results), its age, or whether it is unionized. Adoption is

related to several human-resource management practices, including pay

for skill and profit-sharing or bonus systems, emphasis on training and

skills, desire for a committed work force (and avoidance of contingent

workers), and the prominence of the human-resources management

department. It is not related to hiring and promotion policies, explicit

employment security policies, gain sharing, or wage premiums.

Establishments that adopt flexible work practices differ from those that do

not, but their specific practices do not form identifiable clusters.

Comments This study is one of the few with a nationally representative sample across

virtually all industries, a major strength. However, a high proportion of

responses were from human-resources managers rather than the managers

of production or services to whom the questionnaire was directed. The

effects of flexible work practices on economic or financial performance

were outside the study’s scope.

Page 92 GAO/PEMD-95-28 Schools and Workplaces
Appendix II 

Workplaces: Annotated Bibliography

Management

Processes

Shea and Gobeli John Shea and David Gobeli. “TQM: The Experiences of Ten Small

Businesses.” Business Horizons, January-February 1995, pp. 71-77.

Practice Total quality management.

Source of Information Ten small Oregon businesses—manufacturing, service, and social service

operations—with 18 to 62 employees. Data were collected from a

questionnaire and from interviews with the president or general manager

and employees leading the total quality management implementation

effort.

Findings and Conclusions Five companies had a clearly defined program, two were applying selected

components, and three were using concepts they had learned from books

and seminars and adopted independently. The companies had adopted

total quality management for a variety of reasons, but improving

performance was a recurring theme. They had applied three common

principles: customer orientation, employee empowerment, and continuous

improvement.

All ten organizations thought the strategy was worth the cost. The

principal benefits were improved performance and customer satisfaction.

Although they used many different sources to learn techniques, more than

half hired outside consultants to facilitate training and implementation.

Each organization followed a different implementation process.

Comments While this study suggests that total quality management principles can be

applied to small business with success, the sample size is small and

nonrandom. The results cannot be generalized.

Page 93 GAO/PEMD-95-28 Schools and Workplaces
Appendix II 

Workplaces: Annotated Bibliography

Management

Processes

U.S. Department of Labor U.S. Department of Labor, Office of the American Workplace. High

Performance Work Practices and Firm Performance. Washington, D.C.:

August 1993.

Practice Employee involvement, performance-based compensations, and training.

High-performance work practices typically designed to enhance

employees’ skills, incentives, information, and decisionmaking

responsibility.

Source of Information Twenty-nine studies.

Findings and Conclusions High-performance practices—involving employees in decisionmaking,

tying compensation to a firm’s or an individual’s performance, and

training—are usually associated with increases in the firm’s productivity

but are best implemented together as a system. Employee involvement in

decisionmaking was associated with positive results in 14 studies, negative

in 2, and ambiguous in 13. The link between compensation, specifically

profit sharing, and performance was consistently positive: 3.5-percent to

5-percent higher productivity in 27 studies. One study that examined

formal training programs in 155 manufacturing firms found that those that

introduced a formal program after 1983 experienced a 19-percent greater

rise in productivity in 3 years than firms that did not.

Comments Many studies reported low response rates, often below 50 percent,

possibly indicating an inherent bias in the results. Generalizations should

be made with caution.

Page 94 GAO/PEMD-95-28 Schools and Workplaces
Appendix II 

Workplaces: Annotated Bibliography

Management

Processes

U.S. General Accounting Office U.S. General Accounting Office. Management Practices: U.S. Companies

Improve Performance Through Quality Efforts, GAO/NSIAD-91-190.

Washington, D.C.: 1991.

Practice Formal total quality management practices and how they affect the

performance of selected U.S. companies: what was achieved, how quality

improved, and what lessons may apply to U.S. companies in general.

Source of Information Twenty companies that were among the highest-scoring applicants in 1988

and 1989 for the Malcolm Baldrige National Quality Award.

Findings and Conclusions In almost all cases, companies that used quality-management practices

achieved better employee relations, higher productivity, greater customer

satisfaction, increased market share, and improved profitability. While

each company developed its practices in a unique environment, there were

common features that contributed to their success. These included a focus

on meeting customer requirements; leadership by top management in

building quality values into company operations; training, empowerment,

and involvement for all employees in quality efforts; and integration of

systemic processes to foster continuous improvement throughout the

organization. The benefits of quality management were found in firms

varying in size and economic sector (including both manufacturing and

service industries).

Page 95 GAO/PEMD-95-28 Schools and Workplaces
Appendix II 

Workplaces: Annotated Bibliography

Management

Processes

U.S. General Accounting Office U.S. General Accounting Office. Management Reforms: Examples of Public

and Private Innovations to Improve Service Delivery, GAO/AIMD/GGD-94-90BR.

Washington, D.C.: 1994.

Practice Public and private innovations to improve service delivery, including

changes to the organizational culture (customer focus, results orientation,

and flexible organization), organizational mission (strategic planning,

“core business” focus), organizational structure (partnering,

commercialization, privatizing, franchising, delegation, empowerment, and

regulatory relief and reform), and organizational processes (reengineering,

streamlining, technology applications, benchmarking, and performance

measurement).

Source of Information Literature searches, major clearinghouses, research firms, and other GAO

studies. The report includes an extensive bibliography on management

reforms.

Findings and Conclusions The primary focus of innovation is not in short-term cost or budget

savings. Organizational innovations often require up-front investments of

money and time that may be costly in the short term but are considered

necessary to enhance productivity and avoid future costs.

Organizations became more oriented toward long-term outcomes to

improve performance. Changes typically provided a structure for

measuring and reporting program results, which helped identify programs

or services to be eliminated or improved. Successful restructuring is more

likely to occur when improvement is seen as a long-term process rather

than a “quick fix.”

Successful efforts had four common characteristics: (1) the organizations

used a holistic approach rather than making piecemeal changes;

(2) changes were made only after the organizations had carefully

examined how the changes could be adapted to their particular

circumstances; (3) top management showed commitment and employees

were involved, informed, and free to innovate; and (4) changes typically

allowed managers greater autonomy in exchange for increased

accountability for their performance.

Page 96 GAO/PEMD-95-28 Schools and Workplaces
Appendix II 

Workplaces: Annotated Bibliography

Management

Processes

U.S. General Accounting Office U.S. General Accounting Office. Reengineering Organizations: Results of a

GAO Symposium, GAO/NSIAD-95-34. Washington, D.C.: 1995.

Practice Reengineering organizations.

Source of Information Five executives from manufacturing and service companies cited in the

literature or by experts as engaged in successful reengineering activities.

Findings and Conclusions The symposium developed five principles. (1) Top management must

support and engage in reengineering efforts to remove barriers to success.

(2) An organization’s culture must be receptive to reengineering goals and

principles. (3) Major improvements and savings are realized by focusing

on the business from a process rather than a functional perspective.

(4) Processes should be based on a clear perception of customer needs,

anticipated benefits, and potential for success. (5) Process owners use

teams that are cross-functional, maintain a proper scope, focus on the

customer, and enforce implementation timelines.

Page 97 GAO/PEMD-95-28 Schools and Workplaces
Appendix III 

GAO Contacts and Staff Acknowledgments

GAO Contacts

Schools Gail S. MacColl, Assistant Director (202) 512-5108

Kathleen D. White, Project Manager (202) 512-8512

Workplaces Patrick G. Grasso, Assistant Director (202) 512-5885

Nancy A. Briggs, Deputy Project Manager (202) 512-5703

Acknowledgments In addition to the persons named above, the following staff members made

important contributions to this report: Penny Pickett, Communications

Analyst, and Venkareddy Chennareddy, Referencer.

Page 98 GAO/PEMD-95-28 Schools and Workplaces
Appendix III 

GAO Contacts and Staff Acknowledgments

Page 99 GAO/PEMD-95-28 Schools and Workplaces
Related GAO Products

Teacher Training: Status and Participants’ Views of Delta Teachers

Academy (GAO/RCED-95-208, June 1995).

School Safety: Promising Initiatives for Addressing School Violence

(GAO/HEHS-95-106, April 1995).

Charter Schools: A Growing and Diverse National Reform Movement

(GAO/T-HEHS-95-52, Jan. 1995).

Charter Schools: New Models for Public Schools Provide Opportunities

and Challenges (GAO/HEHS-95-52, Jan. 1995).

Best Practices Methodology: A New Approach for Improving Government

Operations (GAO/NSIAD-95-154, May 1995).

Education Reform: School-Based Management Results in Changes in

Instruction and Budgeting (GAO/HEHS-94-135, Aug. 1994).

Workplace Regulation: Information on Selected Employer and Union

Experiences (GAO/HEHS-94-138, June 1994).

Pay Equity: Experiences of Canada and the Province of Ontario

(GAO/GGD-94-27BR, Nov. 1993).

Competitiveness Issues: The Business Environment in the United States,

Japan, and Germany (GAO/GGD-93-124, Aug. 1993).

Skill Standards: Experience in Certification System Shows Industry

Involvement to Be Key (GAO/HRD-93-30, May 1993).

Technology Transfer: Federal Efforts to Enhance the Competitiveness of

Small Manufacturers (GAO/RCED-92-30, Nov. 1991).

(973801) Page 100 GAO/PEMD-95-28 Schools and Workplaces
Ordering Information

The first copy of each GAO report and testimony is free.

Additional copies are $2 each. Orders should be sent to the

following address, accompanied by a check or money order

made out to the Superintendent of Documents, when

necessary. Orders for 100 or more copies to be mailed to a

single address are discounted 25 percent.

Orders by mail:

U.S. General Accounting Office

P.O. Box 6015

Gaithersburg, MD 20884-6015

or visit:

Room 1100

700 4th St. NW (corner of 4th and G Sts. NW)

U.S. General Accounting Office

Washington, DC

Orders may also be placed by calling (202) 512-6000 

or by using fax number (301) 258-4066, or TDD (301) 413-0006.

Each day, GAO issues a list of newly available reports and

testimony. To receive facsimile copies of the daily list or any

list from the past 30 days, please call (301) 258-4097 using a

touchtone phone. A recorded menu will provide information on

how to obtain these lists.

For information on how to access GAO reports on the INTERNET,

send an e-mail message with "info" in the body to:

info@www.gao.gov

PRINTED ON RECYCLED PAPER
United States

General Accounting Office

Washington, D.C. 20548-0001

Official Business

Penalty for Private Use $300

Address Correction Requested

Bulk Mail

Postage & Fees Paid

GAO

Permit No. G100
GAO/PEMD-95-28 Schools and Workplaces