Measuring What Matters Most

Jeffrey Lucas

Deputy Director, Baldrige National Quality Program

What happens when a K‐12 school system sets a goal beyond meeting the minimum state and federal standards for

student achievement, and instead commits to “Igniting a Passion for Learning”? In the case of Iredell‐Statesville Schools

(I‐SS) in North Carolina, a 2008 Malcolm Baldrige National Quality Award recipient with over 22,000 students and a staff

of over 3,400, the system can produce results like these:

• Reading achievement gaps for both African‐American students and students with disabilities were cut in half.

• In five years, cohort graduation rates increased by 20 percentage points to 81%.

• SAT scores climbed by 60 points in five years to 1056, with increased participation.

• Very low teacher turnover rates and increased applications have meant that 100% of classroom positions have

been filled on the first day of school for the last three years.

• All of this was achieved while I‐SS remained 107th of 115 North Carolina school districts in per‐pupil expenditures

The leaders at I‐SS deserve tremendous credit for their dedication to achieving these types of results, and they in turn

give much credit to the Baldrige National Quality Program’s framework for helping reshape their thinking about what

the “system” part of a local school system really means. This paper looks at one of the most promising features of the

Baldrige framework for K‐12 education—an expanded idea of performance measurement focused on three questions:

why should you measure, what should you measure, and how should you measure it?

Baldrige Program – A Brief Overview

The Baldrige Program, based in the U.S. Department of Commerce’s National Institute of Standards and Technology, was

established by Congress in 1987. The intent was to recognize outstanding performance by organizations in the private

sector as a way to stimulate the competitiveness of U.S. firms at a time when competition from other countries was

strong and growing. Feedback to Congress soon made it clear that a truly competitive U.S. economy would require the

same level of performance excellence in education, health care, and nonprofit organizations. Accordingly, the Program

was expanded to include these sectors. The first recipients of the Baldrige Award K‐12 education were the Chugach and

Pearl River school districts in 2001, followed by Palatine Community Consolidated School District 15 in 2003, Jenks Public

Schools in 2005, and now I‐SS.

Baldrige embraces a three‐part mission focused on improving the competitiveness and performance of all U.S.

organizations. This consists of publishing the Criteria for Performance Excellence, managing the Malcolm Baldrige

National Quality Award based on these Criteria, and serving as an educational program focused on increasing the

capability of organizations to use the Criteria to improve performance.

Using the Baldrige Framework to Measure What Matters

One of the key characteristics of the Education Criteria for Performance Excellence is that they focus on results that

represent an organization’s performance and improvement in all key areas:

• Student Learning Outcomes

• Customer‐Focused Outcomes

• Budgetary, Financial, and Market Outcomes

• Workforce‐Focused Outcomes

• Process Effectiveness Outcomes
• Leadership Outcomes

This performance measurement approach supports a systems perspective on the management of a school or district and

helps align goals across the school or district. Measures serve as both a communications tool and a basis for deploying

consistent overall performance requirements. This alignment ensures consistency of purpose while supporting agility,

innovation, and decentralized decision making.  

Many of the current performance measurement systems used in education rely too heavily on student achievement

data. While the improvement of student learning must remain at the core of what education systems are about,

achievement data alone are not sufficient to drive the types of performance improvements required and do not

acknowledge that schools and districts are in fact complex systems. Recently, thought leaders in education policy have

begun to express similar beliefs. For example, in “Balanced Scorecards and Management Data” (a chapter in the 2009

publication A Byte at the Apple), Ric Hess and Jon Fullerton make a compelling case for expanding the current thinking

about the collection and use of education data to more closely resemble what high‐performing for‐profit organizations

do to make day‐to‐day management decisions and plan long‐term strategy. The Baldrige framework has much to

contribute to such expanded thinking.  

Why Measure?

In the not‐so‐distant past, education organizations suffered from a lack of data on most key areas of their operations. It

was difficult to say with any degree of certainty what students knew and could do, let alone how efficiently the

organization had employed its resources to get students to that level of knowledge and skill or how well such knowledge

and skills were aligned to the expectations of parents and other key stakeholders. With the advent of No Child Left

Behind, the demands for data accelerated exponentially. However, education organizations do not seem to have taken

the time to look beyond the mandated accountability aspect of data to see how a well‐thought‐out system of

measurement, analysis, and knowledge management can form the basis of high levels of performance.

Measures are the key tool that communicates the organization’s important goals to everyone involved and drives

performance toward those ends. The ubiquitous “all the arrows pointing in the same direction” visual indicates the

importance of aligning a school’s or district’s resources so that they are not working at cross‐purposes. Most

practitioners understand that this visual is easier drawn than done. Well‐chosen performance measures can function as

a compass that consistently brings the organization back to its “true north.” What makes devising this compass difficult

is the “well‐chosen” aspect of performance measures.  

Effective performance measures are a key alignment tool in a management system. As such, they must come from a

systematic analysis of what is important to your organization and its stakeholders. The Organizational Profile portion of

the Baldrige Criteria is an important starting point for this systematic analysis. This set of questions asks you to identify

the key influences on how your organization operates and the challenges that you face. It asks you to consider such

questions as “What are your key . . . student segments and stakeholder groups? What are their key requirements and

expectations for your programs . . . and services?” If your answers to such questions identify parents as a key

stakeholder group and children who are “college‐ready” upon graduation as a key requirement, then your performance

measurement system needs to have an effective means of assessing how college‐ready your graduates are. Obviously,

the devil is in the details of constructing an effective measure, but doing so is the only way you will be able to drive

performance that addresses this requirement. Only when everyone in your organization understands that you have an

effective way of keeping score on this dimension will they be able to align their efforts to improve it.
Poorly selected measures can have the opposite effect. They can create organizational friction that takes away from

your ability to improve performance. There is no such thing as a “resource‐neutral” performance measure. In other

words, as soon as you say a measure is important to your organization, resources will begin to flow toward making that

number move. People will begin to allocate increasing amounts of their time, thought, and material resources toward

efforts that they believe can move the number in the desired direction. If you select a measure because it is easy to

collect rather than because you know it is important, you may wind up hampering your progress.  

What to Measure?

To ensure that your selected performance measures support a systems perspective for your school or district, you need

to balance the measures both across the key areas of organizational performance and in‐process as well as end‐of‐

process indicators of performance. The Education Criteria for Performance Excellence encompass four organizational

performance areas. Student Learning Outcomes. Effective measurement of what students know and can do is an area

that has received substantial attention from researchers, policy makers, and legislators. The findings and implications

have been well documented, and to some extent, this is the area where schools and districts have the least latitude in

choosing what to measure. The Baldrige framework places a heavy emphasis on the cause‐and‐effect relationship

between the processes a school or district puts in place and the results it achieves. For this reason, the Criteria

intentionally call for education organizations to report “student learning” rather than “student achievement” results.

The Criteria look mainly for what are currently referred to as “growth models.” That is, the school or district needs to

establish a baseline of individual student achievement at the start of the school year, preferably with reference to

national or state norms; implement its instructional and student support processes; and then measure its success in

terms of its ability to move students upward, both individually and in aggregate, against the same norm‐referenced

group. Many groups, such as the Data Quality Campaign, have been working to ensure the availability of longitudinal

data systems that report high‐quality data at the individual student level to support such measurement.  

The graphic below is from one of the KIPP Academy’s annual report cards. It demonstrates the type of measurement of

student learning called for in the Criteria.
1. Customer‐Focused Outcomes. These results are related to meeting student and stakeholder needs and

expectations. In particular, the Criteria require you to report results for the following:

a. Student and stakeholder satisfaction and dissatisfaction

b. Student and stakeholder relationship building and engagement

As described in “Why Measure?” above, you need to align these measures to your organization’s needs,

accurately and thoroughly identify important student groups and all key stakeholders, and develop a deep

understanding of what their needs and expectations are. Also note the difference between satisfaction and

dissatisfaction, and remember that you need to consider both. While satisfaction is most often measured

through surveys, the respondents frequently do not provide key information about areas where you have not

delivered on their needs or expectations, even if they are not registering strong satisfaction. Information on

dissatisfaction is more often gathered through an effective complaint identification and resolution process, and

the process should have a component that allows key information to be collected, aggregated, and analyzed.  

Another important element to measure is your success in establishing positive relationships with your key

stakeholder groups. Positive relationships are more about what stakeholders will do to support your efforts than

about what stakeholders will say on a survey instrument. Many high‐performing for‐profit organizations have

begun using a “Net Promoter Score” to assess such engagement. By identifying a single powerful question, often

phrased as “How likely would you be to recommend our (product, service, company, etc.) to a friend or

colleague?” and following up on the barriers to respondents who give the top score, organizations have

significantly improved operations and built better relationships in the process.  

2. Budgetary, Financial, and Market Outcomes. These are results related to efficient use of your monetary

resources, as well as to your competitive position in the marketplace. In particular, the Criteria require you to

report results for the following:

a. Budgetary and financial performance, including measures of cost containment or financial viability

b. Market performance, including market share or position, market share growth, or new markets entered

In the first instance, you might report measures such as performance to budget, per‐pupil costs for various

programs, cost avoidance or savings, or allocation of funds to areas of identified importance. You might also

need to do some measurement on the revenue side by looking, for example, at funds received through grants,

awards, or partnerships, or the willingness of the community to provide additional funding through bonds or tax

levies.  

reas of identified importance. You might also

need to do some measurement on the revenue side by looking, for example, at funds received through grants,

awards, or partnerships, or the willingness of the community to provide additional funding through bonds or tax

levies.  
In measuring market performance, public school systems need to be conscious of the increasing options

available to parents and students. They need to view the percentage of parents willing to pay the marginal cost

of a private education as a key indicator of the value that public schools bring to their communities. While

gaining or retaining market share may not be as critical to a public school system as it is for a for‐profit business,

understanding significant shifts in this measure will help you understand the evolving nature of your value

proposition for key stakeholders.  

3. Workforce‐Focused Outcomes. These are results related to meeting the needs and expectations of all segments

of your workforce. In particular, the Criteria require you to report results for the following:

a. Workforce engagement and satisfaction

b. Workforce and leadership development

c. Workforce capability and capacity

d. Workforce climate, including health, safety, security, services, and benefits

With labor representing 70% or more of most school system budgets, your measurement system must cover

both how well you are meeting the requirements of staff and how well you are employing that staff to help

achieve your key outcomes. As in the case of the customer measures described above, the current thinking is

that simple measures of satisfaction may not be sufficient to drive continuous improvement. Unfortunately,

some people are fairly well satisfied when no demands are being made of them. High‐performing organizations

must have an engaged workforce. Engagement is defined as the commitment, both emotional and intellectual,

to accomplish the work, mission, and vision of the organization. It is characterized by people who are motivated

to do their utmost for the benefit of students and stakeholders, and frequently results in spontaneous

innovation and improvement efforts at the most basic levels of the organization. Increasingly sophisticated

means of measuring workforce engagement are becoming available. One of the first descriptions of the impact

of workforce engagement was documented by the Gallup organization in First, Break All the Rules. These more

complex measures are also well supported by more traditional metrics such as turnover rates and absenteeism.

Your school or district also needs to ensure that its human resources meet its needs by measuring both

capability and capacity. These measurements involve finding effective ways to ensure that the individuals who

will deliver the school’s or district’s instructional and support processes have the correct mix of knowledge,

skills, and abilities (capability) as well as that the proper number of these individuals are available to meet the

current demands of the student population (capacity).

4. Process Effectiveness Outcomes. These are results related to the effectiveness and efficiency of your operational

processes. In particular, the Criteria require you to report results for the following:

a. Performance of your work system, including performance of suppliers, partners, and collaborators

b. Performance of your work processes, including productivity, cycle time, efficiency, and innovation

When asked what portions of the Education Criteria are most difficult to respond to, most education users point

to those in the Process Management section. In education, we do not tend to think of what we do as processes.

Dr. Edward Deming, one of the founders of the quality movement, famously said, “If you can’t describe what

you are doing as a process, you don’t know what you are doing.” The process effectiveness measures asked for

in the Criteria will get you to think about your key activities as processes that you can continuously improve.

They will challenge your school or district to become more efficient so that you can reallocate resources that are

currently used less than optimally to your most important activities. Examples of measures might include the

time required to develop curriculum or instructional materials (cycle time) or the number of instructional hours

required per point of achievement gain (productivity).  

Educators often criticize process effectiveness measures as being too “businesslike” and as not respecting the

unique relationships present in the education of children. For this reason, using a balanced set of measures is
important. Balancing your measures ensures that you do not make inappropriate trade‐offs among important

stakeholder groups or in the service of short‐term versus long‐term goals. However, all things being equal, a

school or system that can achieve the same level of student learning in a particular area while consuming fewer

of its scarce resources will be able to devote more of these resources to areas such as innovation and meeting

the customized needs of various student groups.  

5. Leadership Outcomes. These are results related to your system of governance and the performance of senior

leaders. In particular, the Criteria require you to report results for the following:

a. Accomplishment of organizational strategy

b. Governance and fiscal accountability

c. Regulatory, safety, accreditation, and legal compliance

d. Ethical behavior and stakeholder trust

e. Fulfillment of societal responsibilities  

In the leadership outcomes, the Education Criteria expand on the ideas of more traditional scorecards. Some of

the straightforward measures ask you to show that you meet accepted standards for a school or system, such as

accreditation, compliance with state and federal legislation, and adherence to accounting principles. Other areas

are often cited as more challenging to measure, such as accomplishment of your strategy, ethical behavior, and

stakeholder trust. Despite being challenging, this set of measures will be highly important to both stakeholders

and staff. Stakeholders will view these areas as evidence of a fiscally sound, ethically operated organization that

is fulfilling its responsibilities to the community. Staff members will look to these measures to understand how

well they are being led, and there is substantial evidence that when staff members leave, they do not leave

organizations—they leave leaders.

Measures selected in these areas should include both in‐process and end‐of‐process measures of performance.

In‐process measures, which indicate that the process is proceeding as planned, are critical for informing day‐to‐

day decision making. End‐of‐process measures indicate that the overall process has achieved the desired result;

these are important both for assessing the overall performance of the organization and for assessing and

realigning organizational strategy. The concept of in‐process versus end‐of‐process measures is different from

the concept of formative versus summative assessments that schools and districts are familiar with. You collect

and analyze data on in‐process measures at key points in your work processes to determine how well they are

performing. While formative assessments can yield student outcome data that may predict end‐of‐process

performance, they do not provide sufficient information to make operational decisions about improving the

process itself. Those decisions require you to understand the critical inputs—such as time, materials, or

behaviors—and outputs—including quality, quantity, or efficiency—needed to maintain a process at its

optimum level. In more basic terms, giving a formative assessment that demonstrates that students are having

difficulty factoring quadratic equations still doesn’t tell you why this is so or what to do about it. Not every area

of performance will require both types of measures; however, key processes need to be evaluated with both

types when it is practical.

How to Measure?

An effective measurement system under the Baldrige framework uses a number of principles of performance

measurement: looking at current performance and performance trends, selecting and using appropriate comparative

data, appropriately segmenting data, and effectively using performance projections.
1. Performance levels and trends. Your current level of performance is obviously of great importance to you, but in

general, it is best viewed with reference to where the same measure has been in the past. In a system with an

orientation to continuous improvement, you would hope to see an overall positive trend over time.  

2. Use of comparative data. While looking at your performance internally over time can help you understand

whether and at what rate you are improving, it doesn’t tell you how your performance compares with that of

other organizations that are attempting to achieve similar goals. High‐performing organizations use comparative

data toward a number of ends, including evaluating their progress toward targets or goals, communicating their

relative level of performance to stakeholders, and using benchmarking to identify other organizations that have

achieved a desired level of performance in order to study or evaluate those organizations.

Most schools or districts have established goals that they hope to achieve. 2008 Baldrige Award recipient  

I‐SS, for example, had established a goal to be one of the top ten school districts in the state by 2010. I‐SS

routinely displayed its performance against this cohort so that it could evaluate its progress toward this goal in

the time given. (This allowed I‐SS to reach the goal two years ahead of time.) Many districts set long‐term goals

that refer to “world‐class performance.” In many instances, such a performance level can be difficult to identify.

Instead, interim, time‐bounded targets that are more easily measurable may give staff a sense of realistic

accountability. For example, if you set the target “all student segments will be in the top quartile of state

performance by 2012,” you can readily check it for the purpose of evaluating day‐to‐day decisions and

strategies. Such interim results can also be effective communication tools for stakeholder groups. Meeting

interim performance targets relative to the performance of other organizations can serve as a down payment on

a community’s longer‐term investment in world‐class performance for all students. Finally, comparative data

can form the basis of an effective benchmarking effort. In benchmarking, a school or district uses a source of

comparative data to identify the best‐in‐class performers on a particular metric. This information can be helpful

in a number of ways. It can identify what performance level you can realistically achieve on a particular

dimension, and it can provide an opportunity to engage these performers in discussions on the processes they

use to reach these levels. You can then evaluate these approaches for their appropriateness for your own

situation.  

3. Appropriate segmentation. Educational systems are now well versed in segmenting their student achievement

data, usually by the categories required through legislation such as NCLB. You will also need to think about what

segmentation might be appropriate for other measures. For example, at a minimum, you might segment
satisfaction data for parents by building or facility. Numerous other types of segmentation, such as by grade

level, educational program, or student demographics, will also be helpful. Similarly, you would segment

workforce results appropriately by location, job classification, tenure, and other such characteristics.  

Looking at data in aggregate can obscure both opportunities for improvement and role model performance. A

district that looks only at its overall average workforce satisfaction measure of 3.5 on a 5.0‐point scale misses

the fact that Building A is achieving a performance of 4.4 while Building B is only at 2.3. One holds the promise of

best practices to identify and replicate, while the other presents an opportunity for improvement to address in

ways that may not be appropriate for the district as a whole.

4. Use of performance projections. High‐performing organizations have developed effective ways to create valid

estimates of future performance. These projections draw upon past rates of improvement, performance of

comparable organizations, and potential innovations that promise performance breakthroughs. For

management, projections point to potential challenges that your organization faces in achieving a desired goal.

Projections are not targets that specify what is desirable; rather, they represent your best thinking about what is

probable so that you can make appropriate decisions.  

Conclusion

Schools and districts operate in an environment where the rate of change is constantly accelerating. This rapid pace

demands that you optimize performance wherever possible in order to effectively allocate resources to meet the ever‐

changing challenges. To do so, your school or district must understand itself as an interconnected system rather than as

a series of stand‐alone classrooms, departments, or buildings. A well‐constructed set of performance measures is a key

alignment tool that can help you reach this systems perspective on your school or district. The logic model for the

selection and use of performance measures in the Baldrige framework, and its associated Education Criteria for

Performance Excellence, has been refined over more than 20 years through observation of the practices of role‐model

organizations. Wider adoption of this framework by K‐12 districts and schools can lead to the type of performance

breakthroughs achieved by the Baldrige Award recipients in education, including 2008 Baldrige Award recipient Iredell‐

Statesville Schools.

For more information, including serving as a Baldrige Examiner to learn more about the Criteria and how your district or

school can benefit from the Baldrige framework, visit the Baldrige Program at www.nist.gov/baldrige, or call 301‐975‐

2036.