Measuring Up: 

Productivity and Performance in the HOME Program

2003

U.S. Department of Housing and Urban Development 

Community Planning and Development 

Prepared by 

ICF Consulting 

1
Foreword

Developing affordable housing for low-income Americans has become an increasingly complex and challenging job 

for the nation’s states and local jurisdictions. Funding from the HOME Investment Partnerships Program is a 

valuable resource for states and local jurisdictions to design and implement housing programs that address local 

housing needs. The HOME Program provides flexibility to participating jurisdictions, who can choose what types 

of housing programs and activities are most important to meet the housing needs of their low- and very low-income 

residents. 

With these entitlement funds comes an obligation to see that resources are used wisely and effectively. Measuring 

Up: Productivity and Performance in the HOME Program provides guidance and technical assistance to HOME 

Program participating jurisdictions to help them meet this obligation. It provides practical guidance on how to 

measure the productivity and overall performance of HOME-funded programs. Measuring program productivity and 

performance generates important information to program managers about how programs are operating, what is 

working, and what can be improved. In turn, managers can use this information to refine and improve program 

operations and make informed investment decisions. 

Measuring Up is geared to help participating jurisdictions measure program performance and analyze program 

impact from the local perspective. Participating jurisdictions who use these methods to evaluate their programs are 

the most likely to succeed in making sound decisions about how to use HOME funds—and will be able to maximize 

each HOME dollar to leverage the greatest benefit and impact for their clients and communities. 

2
Table of Contents

Foreword 

Introduction ................................................................................................................................................................................. 1 

 Purpose of this Publication................................................................................................................................................... 1 

 Organization of this Model................................................................................................................................................... 2 

 About the Model Guides ...................................................................................................................................................... 2 

Chapter 1: An Introduction to Performance Measurement................................................................................................. 3 

 What is Performance Measurement?.................................................................................................................................... 3 

 Why is Performance Measurement Important? .................................................................................................................... 6 

 Can Performance Measurement Help with Building a High-Performance Team?............................................................... 6 

 Who Needs to Be Involved in Performance Measurement?................................................................................................. 7 

 Using the Program Outcome Model as a Framework for Measuring Performance ............................................................. 8 

 Performance Measurement Methodology ............................................................................................................................ 9 

Chapter 2: Measuring Productivity in Housing Programs................................................................................................. 11 

 What are the Benefits of Measuring Productivity? ............................................................................................................ 11 

 The Productivity Measurement and Analysis Process........................................................................................................ 11 

Chapter 3: Measuring Program Outcomes.......................................................................................................................... 29 

 What is a Program Outcome?............................................................................................................................................. 29 

 What are the Benefits of Measuring Program Outcomes? ................................................................................................. 29 

 The Process of Measuring Program Outcomes .................................................................................................................. 29 

Chapter 4: Bringing It All Together..................................................................................................................................... 39 

 Performance Measurement for Continuous Improvement ................................................................................................. 39 

 Understanding the Relationship Between Program Outcome and Productivity................................................................. 39 

 Case Study 1: Initiating a Performance Measurement System at the State Level ............................................................. 40 

 Case Study 2: Applying Performance Measurement to City’s Rental Housing Program ................................................. 44 

 Case Study 3: Applying Performance Measurement to City Neighborhood Revitalization Program............................... 50 

 Conclusion .......................................................................................................................................................................... 55 

Appendix .................................................................................................................................................................................. 57 

3
Introduction

Purpose of this Publication 

This publication, Measuring Up: Productivity and Performance in the HOME Program, provides guidance and 

technical assistance to HOME Program participating jurisdictions (PJs) that are interested in improving their 

efficiency. Measuring Up describes performance measurement, provides guidance on how to implement a 

performance measurement system, and explains how performance measurement can lead to improvements in the 

productivity of housing agencies. 

Organization and management guru Peter F. Drucker aptly noted, “You can’t manage what you don’t measure, and 

what you don’t manage doesn’t get done.” Performance measurement is a critical management tool that provides 

important information to help managers manage. In other words, performance measurement helps managers make 

informed decisions about program design, staffing, and structural organization in order to meet organizational 

objectives. 

On a state and local level, most housing and community development agencies regularly monitor and report on their 

accomplishments in terms of how much money is spent, how many loans are underwritten, and how many housing 

units are produced. Few agencies consistently examine the relationship between their accomplishments and the 

resources invested in them. Without an analysis of this relationship, it is impossible for a manager to know if 

programs are operating at the most efficient level possible. Still even fewer agencies take the time to measure the 

effectiveness of their programs, in terms of the program’s impact on homeowners, tenants, or neighborhoods. 

Without an analysis of program effectiveness, it is impossible to draw conclusions about the value of and need for 

the programs. 

On the Federal level, the U.S. Department of Housing and Urban 

Development (HUD) has had a longstanding interest in building the 

capacity of its grantees so that they might use productivity 

measurement to manage their funding more efficiently. Foremost, 

HUD wants to see that its funds are used effectively in order to meet as 

many needs as possible. Over a decade ago, HUD’s Office of 

Community Planning and Development issued a notice entitled 

Productivity Guidelines for Community Development Block Grant 

(CDBG) Single Family Rehabilitation Programs.

1

 This publication 

was issued in 

response to concerns about the excessive administrative cost 

and limited production of CDBG-funded single-family 

rehabilitation programs. The notice provided national 

“benchmarks,” or targets, against which grantees could 

measure their program efficiency (e.g., cost per unit 

completed). Measuring Up moves beyond those early national 

benchmarks to guide communities in thinking about locallybased measurements against which to monitor and gauge 

performance. 

The Federal government continues to press for efficiency in 

program administration. In 1993, Congress passed the 

Government Performance and Results Act (GPRA). This Act 

directly links Congressional decision-making on program 

4

“You can’t manage what 

you don’t measure, and 

what you don’t manage 

doesn’t get done.” 

— Peter F. Drucker 

Performance measurement helps managers 

avoid difficult situations like these:

In a public hearing, a council representative asks 

why the average development cost of one unit of 

housing is $110,000 in the city’s affordable housing 

program, while it is only $80,000 in the private 

market. 

An inspector on staff does not complete nearly as 

many inspections in a week as the manager suspects 

he should be able to. The manager does not know 

whether the employee has a heavy workload, or if 

he is not efficient in his work. 

A CDC has an unexpectedly high drop-out rate in 

its homebuyer assistance program, and it does not 

know why or what to do about it. It risks recapture 

of funds by the PJ. 
spending to the effectiveness and efficiency of programs in achieving statutory objectives. GPRA emphasizes: 

• Program effectiveness and results; 

• Increased service quality and customer satisfaction; and 

• Improved public accountability. 

With the passage of GPRA, it became more important than ever before that HUD be able to articulate the 

accomplishments of the Department and its grantees. To a large extent, the future of HOME Program funding is 

directly linked to the productivity and effectiveness of its PJs. 

Since PJs are given flexibility to make choices about how to use HOME funds, it is only logical that they be held 

accountable for those choices. Measuring Up provides technical advice to help PJs use performance measurement 

to take the guesswork out of programmatic decisions. This publication assists PJs in defining organizational 

problems, developing a methodology for implementing performance measurement within their organizations, 

analyzing the efficiency and effectiveness of their programs, and generating and using local benchmarks that 

consider local housing goals, as well as other economic and political factors, to monitor performance over time. 

Organization of this Model 

This model guide is written primarily for HOME Program managers. It is organized as follows: 

Chapter 1: An Introduction to Performance Measurement addresses the most basic of questions: what is 

performance measurement, and how can it help program managers and staff achieve greater results in their 

programs? The chapter presents a theoretical framework for understanding issues related to performance 

measurement, and describes several important benefits of measuring performance. 

Chapter 2: Measuring Productivity examines ways to use measurement as a tool for improving productivity 

within local programs. The chapter provides step-by-step guidance in how to measure productivity and efficiency 

in program operations, including identifying problems, assessing data collection needs, implementing data 

collection procedures, and analyzing findings. By analyzing key factors, such as the time required to complete each 

step in the production process, the relationship between steps, and the accuracy of the work produced, PJs can often 

identify practical steps that might result in increased production of units or more timely expenditure of program 

funds. 

Chapter 3: Measuring Program Outcomes goes one step further and focuses on ways to examine the outcomes of 

programs—that is, the benefits programs are producing for program participants and the community as a whole. 

Given relatively flat funding rates but ever-growing community needs, it is important for program mangers and staff 

to be able to compare outcomes against program goals to ensure that HOME funds are being used strategically to 

address the most pressing needs within a community. This chapter provides specific guidelines on how to measure 

program outcomes for a variety of HOME-eligible housing activities. 

Chapter 4: Bringing It All Together summarizes the key lessons of the model, and provides a vision for how 

combining productivity and outcome measurement can keep PJs focused on their affordable housing goals. 

Comprehensive case studies are incorporated within the chapter to illustrate the application of the principles in the 

publication. 

About the Model Guides 

This publication is one of a series of model guides published by the Office of Affordable Housing Programs of the 

U.S. Department of Housing and Urban Development. The model guide series offers technical assistance and 

practical guidance to PJs in HOME Program implementation. To get a free copy of this model program guide, see 

HUD’s Office of Affordable Housing Programs online library at 

http://www.hud.gov/offices/cpd/affordablehousing/ library/modelguides/index.cfm.

End Note 

5
1 U.S. Department of Housing and Urban Development, “Productivity Guidelines for Community 

Development Block Grant Single Family Rehabilitation Programs,” CPD-90-12, March 9, 1990. 

Chapter 1: 

 An Introduction to Performance Measurement 

In the context of growing needs, absent growing resources, housing program managers must find a way to 

get more out of every dollar allocated to their programs. Performance measurement is a process of data 

collection and analysis used to improve organizational results. When it is used judiciously and 

strategically, it can help managers improve the efficiency and effectiveness of their programs. 

This chapter provides an overview of performance measurement, including a description of productivity 

and program outcome measurement, and explores why measuring performance is important. The chapter 

also presents an overview of the program outcome model, and describes basic measurement methodology.

What Is Performance Measurement? 

Performance measurement is an organized process used by managers to gather information to determine how well 

programs and projects are meeting needs, and then to use that information to improve performance and target 

resources more precisely. There are two critical components of performance measurement: (1) productivity; and (2) 

program impact (or outcome). 

In housing programs, productivity reflects the level of efficiency (quantity, quality, and pace) with which a 

participating jurisdiction (PJ) undertakes its activities. Program impact, on the other hand, reflects the extent to 

which those activities yield the desired outcomes in the community. Typical “desired outcomes” for affordable 

housing programs include neighborhood revitalization, improved quality of life for participants, or an increase in the 

affordable housing stock. 

Productivity 

Productivity is the relationship of resources invested (inputs) to the quantity and quality of what is produced 

(outputs). Typically, productivity measures help a program manager answer the question, “What is the organization 

producing?” Most housing organizations use the basic housing production measures identified in Figure 1.1 below. 

Figure 1.1 

Standard Output Measurements in HOME Programs 

Program Standard Measurements (Outputs) 

Rehabilitation programs 

New construction programs 

Tenant-based assistance 

Number of housing units rehabilitated 

Number of housing units constructed 

Number of households assisted 

programs 

Homebuyer assistance Number of households assisted 

programs 

These simple measures are measurements of program outputs—the actual program accomplishments. Measuring 

program accomplishments is useful to program managers because the measurements provide a way to define how 

much work the agency has done. For programs with established goals, these measurements also help managers 

determine whether or not the agency is meeting its goals. Finally, if accomplishments are tracked from year-to-year, 

6
managers can compare how much work is done from one year to the next, and make assessments about the agency’s 

progress. 

Measuring Productivity: The Thomastowne Case Study 

Imagine you are the newly hired housing director of the Thomastowne Housing Division. Traditionally, Thomastowne 

has had a strong rehabilitation program, but had no experience undertaking a new construction activity. The previous 

director, at the mayor’s urging, had invested the agency’s HOME funds in a new construction development at a strategic 

site in town. The ribbon-cutting ceremony for the project is held your first week on the job. Fifteen well-designed homes 

beam in the sunshine. The mayor and neighborhood residents applaud the agency’s accomplishment. You, too, are very 

impressed with what you see. 

On your way back to the office, you talk to the project manager of the job. She expresses some concerns about the 

marketability of the units, as only two of the fifteen units have been sold. You wonder how this could be, and decide to 

review the project files. In your review, you learn three disturbing facts: 

• The project costs were unreasonably high: the units cost $175,000 to construct, the appraised value is $160,000, 

and the area median is only $120,000. The units were sold for $100,000. 

• The construction process was unexpectedly long: the project was initially approved for funding seven years ago. 

• A key document, a market study, was never conducted to assess the marketability of homeownership units at this 

location.

You know you have substantial work ahead. 

Most program managers understand the need to measure program outputs, and rely on this data to inform others 

(such as elected officials and neighborhood residents) about the agency’s accomplishments. 

As the Thomastowne case study (see page 4) illustrates, the measurement of accomplishments only (in this case, 

fifteen units of newly constructed housing) can be misleading when the accomplishments are not evaluated in 

relation to resources (time and money). At a time when most agencies find that their financial and staff resources 

are insufficient relative to housing needs in their jurisdictions, it is critical that managers examine productivity in 

terms of program accomplishments relative to resource expenditure. For most agencies, staff time and housing 

dollars are the most significant resources to measure. Tracking tools and reporting systems for staff can help 

managers track data on outputs and resource expenditure on an ongoing basis. By regularly reviewing production 

data, managers can flag and identify problems before they become insurmountable. 

Arguably, it is even more critical to evaluate the impact of the programs on the overall housing program goal(s), 

known as program outcomes measurement. 

Program Outcomes Measurement 

Whenever resources are scarce, it is essential to know where program activities are making a difference and where 

they are not. The purpose behind outcome measurement is to help ensure that programs address the identified needs 

within a community as effectively as possible. 

Program outcomes are generally defined as the benefits or results of a program. Typically, program outcomes 

relate to a change in condition, status, skills, knowledge, or behavior. Sometimes, however, program outcomes 

involve helping individuals to maintain their condition or status. 

Figure 1.2 provides several examples of outcomes of housing programs. 

7
Figure 1.2 

Examples of Housing Program Outcomes 

Type of Change Example in a housing program 

Change in 

circumstance 

Neighborhood is revitalized. 

Change in status Renter becomes a homeowner 

Change in behavior Prospective homeowner pays bills on time to improve 

credit history 

Change in functioning Disabled clients are housed in units that increase their 

independence 

Change in attitude Perception of neighborhood from unsafe to safe 

Change in knowledge Understands the steps involved in becoming a 

homeowner 

Change in skills Able to do general home maintenance/repair 

Maintenance Elderly can remain in their homes 

Prevention Quality of housing stock does not deteriorate 

Housing program outcomes typically fall into one of two general categories: (1) benefits to program participants or, 

(2) benefits to neighborhoods. Some benefits to program participants (the individuals and families who move into 

the housing) include: 

• An improved housing unit/living space; 

• An increased sense of stability; 

• An opportunity to build economic assets; and/or 

• Lower monthly housing payments. 

Benefits to others in the community might include: 

• A decrease in the number of deteriorating structures in the neighborhood; 

• An increase in property values in the immediate neighborhood; 

• Improved safety and desirability of the neighborhood; and/or 

• An improved environment for local businesses. 

Performance measurement must be viewed in terms of productivity and program outcomes together, and 

performance measurement results are generally most valuable when evaluated in terms of the community’s own 

needs. 

Measuring Program Outcomes: 

Smootsville Case Study

The City of Smootsville administers a rental housing development program designed to create more affordable units for occupancy

by low-income families. It reports its housing production to the mayor on a quarterly basis. Last year, construction costs in the area 

skyrocketed. Rather than lower its production goals, however, City staff decided to offset the cost increases by developing the 

affordable units in the neighborhoods where property values are the lowest in the city. 

Under the modified plan, Smootsville was able to meet its cost and production goals. However, the units were not marketable 

because of the undesirability of the neighborhoods in which they were built. Since the program goal is to provide a greater number 

of units for occupancy by low-income families, the program did not have the intended impact within the community. As 

Smootsville staff came to realize,the construction of new rental units offers little value in and of itself until the units are leased. And 

to be leased, the units must be of suitable quality (including size), in a desirable location, and affordable to the target population. 

8
Strategic Use of Performance Measurement 

It would be an all-consuming effort for a housing agency to measure nearly every aspect of its performance. PJs 

need not measure everything. Measurement should be used strategically, and on an ongoing basis, to answer certain 

fundamental questions, such as: 

• Will HOME dollars be committed and dispersed by the statutory/regulatory deadlines? 

• Are CHDOs, state recipients, or other housing partners meeting agreed upon milestones and production 

levels? 

• What is the development cost per unit? 

• Is the workload distributed fairly among staff? 

• How much time elapses between the point of application and completed product? 

• How much private money is leveraged in HOME projects? 

• Is the target population (in terms of clients, neighborhoods, or houses) being served? 

• Are the program outcomes—not just the production goals—being reached? 

When a program’s performance in response to these big picture questions 

is not meeting a manager’s expectations, or when the PJ’s results differ 

significantly from neighboring PJ’s results, measurement can be used to 

help a manager understand the reason(s) for this unexpected performance. 

Why Use Performance 

Measurement?

• Stretch HOME dollars further to 

meet more needs. 

• Communicate program 

accomplishments and build support 

for program(s). 

• Inform program design, 

implementation, and reporting.

• Build a more motivated and 

effective team. 

Similarly, performance measurement can be applied to entire programs to 

track and monitor progress. For example, a state might use performance 

measurement to track and evaluate its state recipients, or a local PJ might 

use it to manage a subrecipient or community based housing development 

organization. The PJ would establish a range of benchmarks that it 

expects its housing partner to meet. When the partner fails to perform to 

the established benchmark, the PJ can use this information to impose 

sanctions and/or provide technical assistance in order to guide the partner 

back on track. PJs can also use performance data to inform future funding 

decisions. 

Why Is Performance Measurement Important? 

There are a number of reasons why program managers should use performance measurement as a tool: 

• Performance measurement can help stretch a program’s HOME dollars further. While HOME funds 

are significant, they are not infinite. By using performance measurement to evaluate an agency’s 

productivity and effectiveness, managers are better able to target funds to activities that have the most impact, 

thereby stretching HOME dollars to assist more low-income households. 

• Performance measurement can lead to better decisions about program design and implementation. By 

using performance measurement to identify and diagnose problems, PJs can solve their production problems 

more effectively. This can lead to improved decisions related to: 

— Program design (who the program serves and does not serve; how much assistance is available; what 

neighborhoods are targeted); 

— Personnel (matching the right people to the right jobs); and 

— Process (designing procedures that are as effective and efficient as possible). 

9
• Performance measurement generates data that can simplify reporting. A systematic approach to 

reviewing program performance can allow for better-informed long-range planning. PJs that are able to 

collect and analyze data on an ongoing basis are well-prepared when it is time to complete Consolidated 

Plans and Consolidated Annual Performance Evaluation Reports (CAPERs). 

• Performance measurement can help managers build a more effective and motivated team. Performance 

measurement is key to ensuring that workloads are balanced and that individuals are not overwhelmed. It can 

also help reduce or eliminate inefficiencies that make employees’ jobs needlessly frustrating. In addition, 

performance measurement can be 

used to create a system of 

accountability that allows project 

managers to delegate more work 

and allows staff to take more 

initiative. By measuring 

performance, managers can 

identify and reward success. 

Finally, a stronger understanding 

of the impact a program is having 

on the community and the lives 

of the individuals served may 

motivate staff and foster pride in 

the program. 

• Performance measurement can 

help communicate accomplishments and build support for a program. With data analysis, PJs are able 

to communicate to investors, public or private, that their money is being used efficiently, effectively, and for 

its stated purpose. HUD’s own use of performance data supports its requests to the congressional budget 

committees. 

Sample problems that can be solved with performance 

measurement:

• Determine the cause of a decline in the number of 

applications for the agency’s tenant-based rental assistance 

program; 

• Find solutions to a lengthy proposal review time for 

development proposals received in response to an RFP. 

• Determine whether or not a program’s subsidy limits are 

appropriate for the target beneficiaries. 

• Determine whether or not a neighborhood revitalization 

program is meeting its goals. 

Can Performance Measurement Help With Building a HighPerformance Team? 

A truly effective organization relies on a solid, high-performance team. It is hard to build an effective team without 

a solid performance measurement system in place. Figure 1.3 on the following page illustrates the variety of ways 

that measuring performance can impact the effectiveness of an organization. 

10
Figure 1.3 

How Performance Measurement Supports High-Performance Teams 

What managers can do if they have a performance 

measurement system in place: 

Characteristics of 

High-Performance 

Team 

❑ Distribute work more effectively and fairly 

because they know the actual workload of staff 

members. 

❑ Identify when a staff member is much less 

productive than co-workers, which might 

indicate a poor job match. 

❑ Delegate work more effectively and ensure 

accountability because staff is better informed. 

The right people 

doing the right jobs 

❑ Understand the impact of the agency’s work in 

the community and instill a sense of pride 

among staff. 

“Can-do” team 

culture 

❑ Make decisions based on facts (rather than 

perceptions), and build staff trust and respect. 

❑ Delegate more and micromanage less because 

there are standards to which staff can be held 

accountable. 

Trust, respect, and 

appreciation 

between staff and 

management 

❑ Establish and communicate precise 

expectations, and evaluate staff more 

objectively. For example, measurement is the 

difference between “You must be more 

productive,” and “I need you to close 15% more 

loans this year.” 

Clear performance 

expectations 

❑ Identify and respond to success and poor 

performance. Apply incentive programs and 

disciplinary actions fairly and consistently. 

(Having good data minimizes guesswork.) 

Performance is 

measured and 

rewarded and poor 

performance is 

handled as 

appropriate 

❑ Communicate progress and gather input based 

on clear, practical, easily understood 

measures. 

❑ Use a common expectation against which 

progress can be understood and 

communicated to constituents and 

stakeholders. 

Effective internal 

and external 

communications 

about program 

success 

❑ Make structural changes to work systems with 

confidence because decisions are based on 

accurate information on how the process is 

currently working. 

Program/process 

improvement 

❑ Determine where training and technical 

assistance should be targeted, based on 

measured staff needs. 

Training and quality 

technical 

assistance 

11
Who Needs to Be Involved in Performance Measurement? 

Even though getting everyone involved in performance measurement is neither easy nor practical, being inclusive in 

the development of a performance measurement system is essential. The type and extent of involvement should be 

based on the performance measurement effort being developed and undertaken. Consider involving the following 

groups of people: 

• Staff. Any performance measurement system must include staff at all levels. Staff has the best information 

about what is happening “on the ground,” and therefore is best able to identify and diagnose problems. 

Moreover, once problems are identified and new strategies developed, it is the staff that carries primary 

responsibility for implementation of new procedures. Any change in routine can be challenging and 

potentially threatening for staff, especially for public employees who operate in an environment where every 

action is open to public scrutiny and where most information is subject to “sunshine” laws. It is therefore 

critical to involve staff—particularly middle management and front-line staff—in the design and 

implementation of the system. It is also essential to involve top management or the day-to-day demands of 

the program operation may overcome the task. Before they will commit to measurement, however, agency 

directors and board presidents will need to understand its value for their agencies. 

• Political and community leadership. First and foremost, performance measures are designed to assess 

program achievement (productivity and outcomes). It is important that political and community leadership 

support the measurement effort, and that the measures capture the goals of these groups. 

• Business partners. When using measurement to undertake improvement efforts, it may become important to 

involve the PJ’s business partners, such as builders, contractors, real estate agents, social service agencies, 

appraisers, and lenders. These partners will be able to provide valuable information about the 

interrelationship between the agency’s performance and their business needs and capacity. 

• Clients and potential clients. Particularly when evaluating program outcomes, it is important to gather and 

consider feedback from clients and potential clients. This group will likely be very diverse and should 

include clients who are served by the program(s), potential clients who apply but are not served, and potential 

clients who are eligible for the program but do not apply. 

Using the Program Outcome Model as a Framework for 

Measuring Performance 

For the purpose of this publication, performance measurement is perhaps best 

explained using the “program outcome model.” The program outcome model 

demonstrates the relationship between an organization’s inputs, activities, 

outputs, and outcomes. 

• Inputs include resources dedicated to, or consumed by, the program. 

Examples include money, staff and staff time, equipment, and supplies. 

Inputs also include constraints on the program, such as laws, regulations, 

and requirements or conditions for receipt of funding. 

• Activities are what the program does with inputs to fulfill its mission. 

Activities include the strategies, techniques, and types of treatment that 

comprise the program’s production process or service methodology. 

Examples include marketing, screening applicants, processing loans, conducting inspections, etc. Program 

activities result in outputs. 

The program outcome 

model demonstrates 

the relationship 

between inputs, 

activities, outputs, and 

outcomes. 

• Outputs are the direct products of a program’s activities and are usually measured in terms of the volume of 

work accomplished. Common outputs of housing programs include number of low-income households 

served, number of loan applications processed, or number of units constructed or rehabilitated. The sum of a 

program’s outputs should result in positive outcomes for the program’s participants. 

12
• Outcomes are benefits for participants during or after their involvement with a program. Outcomes typically 

relate to a change in condition, status, attitude, skill, knowledge, or behavior. Examples of outcomes for 

housing programs include improved quality of life for program participants, increased housing stability, 

improved quality of the local housing stock, or revitalization of a neighborhood. 

Most PJs monitor their inputs (e.g., resources invested) and outputs (e.g., units produced), but few examine the 

relationship between the inputs and outputs. It is only through this assessment of inputs relative to outputs that a PJ 

can know if its program is operating at maximum efficiency, or, if activities need to be modified. However, 

understanding program productivity is not enough, for it does not indicate whether resources are being targeted 

towards the right activities. As a result, PJs also need to measure outcomes. In other words, it is the examination of 

inputs relative to outputs that indicates whether a PJ is “doing things right,” but it is the examination of outcomes 

that indicates whether a PJ is “doing the right things.” Both are necessary for high performance. 

Figure 1.4 illustrates the program outcome model as it applies to an owner occupied rehabilitation program. 

Figure 1.4 

Program Outcome Model for an Owner-Occupied Rehabilitation Program1

INPUT 

Resources 

dedicated to or 

consumed by the 

program 

• Money 

• Staff/staff 

time 

• Contractors 

• Facilities 

• Equipment 

Constraints on the 

program 

• Laws 

• Regulations 

• Funders’ 

requirement 

ACTIVITIES 

What the program 

does with the inputs

to fulfill its mission 

• Intake/loan 

screening 

• Initial 

inspection 

• Verify 

contractor 

eligibility and cost 

reasonableness 

• Prepare 

construction 

specifications 

• Underwrite 

loan 

• Loan 

approval 

• Progress 

inspections 

OUTPUTS 

The direct 

products of 

the program 

activities 

• Number of 

customers served

• Number of 

loans processed

• Number of 

homes 

rehabilitated

OUTCOMES 

Benefits for 

participants as a result 

of their involvement in 

the program 

• Increased 

financial stability for 

program participants

• Increased 

housing stability for 

program participants

• Revitalized 

neighborhood

• Improved 

quality/durability 

housing stock

Performance Measurement Methodology 

The process of measuring performance in a program is not necessarily a linear one, particularly for organizations 

that already have some level of performance measurement in place and want to improve existing efforts. However, 

it is useful to break down the program outcome model into a series of sequential steps that includes: 

• Identifying and defining the problem; 

• Determining information needs and identifying data sources; 

• Measuring and tracking data; 

• Analyzing data and identifying areas for improvements; 

• Implementing improvements; and 

• Continuing measurement of progress. 

13
Figure 1.5 illustrates how the program outcome methodology works. However, organizational needs vary 

considerably in terms of how much detail needs to be undertaken at each of these steps. For example, some 

organizations already collect and track data, but need to analyze it more regularly to identify needed improvements; 

in contrast, other organizations need to improve their data collection and tracking methods. This publication 

presents all the components of each step, as if an organization is implementing a performance measurement system 

for the first time. While this may seem overwhelming, it is expected that PJs will use the components of the process 

that best complement and improve their existing efforts. PJs need not undertake every activity described in this 

publication to see results. Doing even one or two things can have a positive effect on a program. 

Figure 1.5 

Performance Measurement—A Model for Continuous Improvement 

Step1:

Identify Performance Concerns/Goals

Step 1:

Step 2:

Identify Information Needs/Select Tools

Step1:

Identify Performance Concerns/Goals

Step 3:

Apply Measurement Tools

Step 4:

Analyze Results

Step1:

Identify Performance Concerns/Goals

Step 5:

Implement Improvements

Step 6:

Continue Measuring Performance Measure

Assess

14
Chapter 2: 

 Measuring Productivity in Housing Programs 

Measuring productivity is at the core of performance measurement because it specifically defines how 

much an agency accomplishes. Chapter 2 explores what it means to measure the productivity, or 

efficiency, of program operations and provides a methodology for solving problems through productivity 

measurement. Chapter 2 also provides guidance on how to incorporate productivity measurement into 

existing production processes. 

What are the Benefits of Measuring Productivity? 

Productivity measurement and analysis, although only one piece of an overall performance measurement system, 

can by itself be undertaken to help managers solve problems related to the program’s purpose, process, schedule, 

efficiency, and to some extent quality. In other words, productivity measurement and analysis helps program 

managers: 

• Increase production without increasing resources. Even though the organization might be meeting its 

goals, it might be able to operate more efficiently. Breaking down activities may show steps that are 

duplicative or otherwise unnecessary; it may also reveal how activities can be streamlined. This, in the end, 

will increase the organization’s production and make it more effective at meeting community needs. 

• Identify bottlenecks that are slowing down program activities. Analyzing the production process can 

help identify progress toward milestones, and plan for meeting goals more effectively. For example, by 

analyzing the production process, a manager can estimate when construction must begin on each unit in order 

to meet an annual rehabilitation goal of twenty units. 

• Hold CHDOs, recipients, subrecipients, and employees accountable for their work. Focusing on the 

individuals responsible for and involved in each phase of a program process, and monitoring each person’s 

output, often results in greater accountability since “what gets measured, gets managed.” 

• Chart progress against benchmarks to evaluate performance. If a CHDO, state recipient, workgroup, or 

an employee generates a certain level of production in one year, recording that level and tracking the 

production over time can assist a manager in determining whether or not the entity is working at capacity 

each year. 

• Identify root causes of problems the agency may be experiencing, not symptoms. It is only through a 

careful and close examination of the production process, including a breakdown of each step in the process, 

that a manager can gain insight into the cause of a problem. This type of analysis leads to the identification 

of an appropriate solution(s) to the problem. For instance, a cursory examination of an organization’s 

production might reveal that it takes anywhere from two to forty days to schedule an initial inspection in a 

homeowner rehabilitation program. This variation in time might be caused by any number of things—an 

overworked inspection staff that does not have time to schedule new inspections; an inspector(s) that is not 

meeting his/her responsibilities; the lack of clear procedures or assignments that result in random, rather than 

systematic, scheduling of inspections; or clientele that is unavailable by phone. 

• Identify staff training needs. Findings can indicate areas where staff is having special difficulties, enabling 

the manager to secure targeted technical assistance or training. 

15
The Productivity Measurement and Analysis Process 

The process of measuring productivity in a program can be divided into six steps. This publication focuses on the 

first four of these steps: 

1. Identify performance concerns or goals to address through measurement. 

2. Identify needed data and determine potential data sources. 

3. Apply measurement tools. 

4. Analyze the data and identify areas for improvements. 

5. Im plement improvements. 

6. Continue measuring performance. 

Step 1. Identify Performance Concerns/Goals to Address Through Measurement 

Incorporating productivity measurement into programs involves time and thought. Careful planning in this first 

step, identifying concerns and goals, greatly facilitates the remainder of the process. This identification process 

should be done somewhat systematically, with two specific objectives: 

• Obtain a “big picture” perspective on the issues; and 

• Map out the production process and identify key milestones for the program(s) that is(are) to be targeted. 

A “Big Picture” Assessment 

Before conducting a more detailed review of program operations, managers need to make some decisions about 

where to focus their energies. Assessing the big picture status of the agency’s programs is a good starting point. 

Two tools to help managers make this assessment include: (1) observing program operations; and (2) examining 

program expenditure levels. 

Observing Program Operations

Eyes, ears, and common sense may be the best tools a manager has at his or her disposal. Managers who pay 

attention to what is going on in their office can generally learn about the workflow of programs. For instance: 

• Are some staff members busy (and/or harried), but others are not? Is there any regularity to these extremes? 

• Are there staff members whose jobs require significant coordination with other divisions in the agency who 

are always in their own workspace? 

• Is there any physical evidence that might provide clues about the productivity of the agency? For instance, 

do some staff members have big piles of files overwhelming their workspace? 

• Are staff and other stakeholders complaining about the program (for example, coordinating with other 

divisions or entities, or the quality or quantity of work produced)? Inside and outside sources of complaints 

can be a very good indicator of problems that warrant more examination. 

At this stage, it is too early in the process to draw any conclusions from observation. For example, it would be a 

mistake to think that just because one staff person has many files on his or her desk and another does not that one is 

more productive. Likewise, staff who always appear busy could be very disorganized or simply overworked-or 

both. In other words, observation alone will not lead to an understanding of the cause of a particular problem. It 

can, however, help an astute manager “flag” programs or issues that require more examination. 

Once work flow within the agency has been observed, it is important to begin the process of looking at whether or 

not there is any correlation between those observations and the production information the agency does have. There 

is one source of production data that is available to all PJs-HUD’s Integrated Disbursement and Information System 

(IDIS). 

16

Using IDIS data 

IDIS data is only helpful if it is accurate and complete. If 

the agency’s IDIS data is not reliable, the agency should 

address this problem before embarking on any additional 

performance measurement efforts. 

IDIS Technical 

Assistance Hotline 

1.800.273.2573 
Basic Production Levels 

IDIS provides PJs with an indication of how well a program is functioning. IDIS provides key baseline information 

on basic program indicators, including program commitments, disbursements, and total units produced. 

There are four IDIS tools that are particularly helpful when it comes to monitoring productivity: 

• The HOME SNAPSHOT. Based on data entered into the IDIS system, HUD generates a production report 

for every HOME PJ. These reports are made available on-line at www.hud.gov/offices/ 

cpd/affordablehousing/programs/ home/snapshot/index.cfm and show PJs how their production numbers 

compare to other PJs. 

 The SNAPSHOT is based on a variety of regulatory and policy factors that HUD considers important 

measures of a PJ’s productivity, detailed in Figure 2.1. PJs will benefit from collecting additional data to 

glean more detailed information on some of the measures identified by HUD. Alternately, PJs may identify 

entirely new data to collect in order to measure production in additional areas that are important at the local 

or state level. The HUD SNAPSHOT and any additional local report with similar production data is the most 

logical starting point for a big picture assessment of program productivity. 

 The SNAPSHOT identifies: 

— Data that, when taken together, give a picture of the jurisdiction’s overall productivity/performance. 

— For local jurisdictions, performance data from all PJs within the state, all PJs within a similar funding 

level, and all PJs nationally. This data can be used as a benchmark for a PJ to assess relative 

performance in a number of areas. 

— For states, performance data from all other states. This data can be used as a benchmark for states to 

assess performance in a number of areas relative to other state PJs. 

• Web-based HOME Production Reports. Each month, HUD posts Commitment, CHDO Reservation, and 

Disbursement Status Reports on its website to help PJs monitor their compliance with HOME statutory and 

regulatory requirements for committing funds within two years of receipt and expending them within five 

years. To view these reports, visit HUD’s website at 

http://www.hud.gov/offices/cpd/affordablehousing/reports/index.cfm. 

• PR27 (Status of HOME Grants Report). This report is generated by the PJ and contains basic financial 

information on HOME grants, subgrants, and subfunds, including commitments, disbursements, 

administrative funds, and CHDO reservations. This report answers the following questions: 

— Are all HOME funds two years or older committed? 

— Are all HOME funds five years or older disbursed? 

— In more recent years, are appropriate levels of funds committed and expended? 

— Are administrative funds within the cap? 

— Does the CHDO reservation meet minimum requirements? 

• PR22 (Status of HOME Activities Report). The PR22, generated by the PJ, provides the status of HOME 

activities on a project-by-project basis. This report answers the following questions: 

— Are any projects stalled? If so, which ones? 

— Are stalled projects associated with certain developers, CHDOs, or state recipients? 

— Are there projects that are complete but remain “open”? If so, the project may be unnecessarily tying up 

funds that could be used for other projects. 

— Does there appear to be a problem with untimely entry of commitments into IDIS? This could signal a 

backlog of work among specific CHDOs, state recipients, or employees. 

— Does there appear to be a problem with untimely entry of completion reports? This could signal a 

backlog of work or a problem with the project. 

— What do commitment dates indicate about program operations? Large lapses in time between the initial 

commitment of funds to project completion could signal a problem within the agency. 

17
18
Analyzing a HUD SNAPSHOT

In general, the PJ described by this HUD SNAPSHOT appears to have a very strong program. It is ranked first in 

its state, and is in the 85 percentile or higher for five of the eight performance indicators established by HUD. 

An analysis of the data provided by this SNAPSHOT, however, identifies some possible issues that an effective 

program manager would explore further: 

1 Relatively little of the PJ’s HOME funds are directed at rental activities. This issue should be 

evaluated to determine if the community's need for affordable rental housing is being met by some other 

program offered by the PJ. If not, program investment decisions may need to be re-evaluated by this PJ. 

2 Low-income benefit is limited. The PJ’s rental housing program is serving a relatively low number of very 

low-income households (whose incomes are at or below 50 percent of area median income). Of those lowincome households, however, households with extremely low incomes (at or below 30 percent of area 

median) are being reached somewhat more effectively. This limited low-income benefit may indicate a 

potential compliance problem with the HOME requirement that 90 percent of funds used for rental housing 

be invested in households at or below 60 percent of area median income. 

3 Lease-up of rental units is lagging far behind completion of units. The data indicate that nearly 25 

percent of the completed rental units are not yet occupied. This problem might be related to a number of 

things, and the following questions must be answered to understand the issue: 

a. Is occupancy data input into IDIS in a timely manner? If not, is this a personnel or procedural problem? 

b. If units are not occupied upon completion, is there a market for affordable rental units in the 

neighborhoods where investments are being made? Is this a policy/procedural problem (was a market 

study ever conducted?) or a program design problem (investment was made even though market study 

suggested there was no demand for housing in that location)? 

c. Is there a program administration problem that might be causing delays in rent-up? If so, is the problem 

related to personnel, procedures, or both? 

4 Rental development costs are notably below average for the state and the nation. To determine the 

actual rental cost per unit, the HOME rental cost per unit (in the cost section of the SNAPSHOT) must be 

multiplied by the leveraging ratio provided in the SNAPSHOT (Program Progress section). The data raise 

the following questions: 

a. Is the cost data accurate? If not, is this a personnel or procedures problem? 

b. If the data are accurate, it might suggest that the rental units selected for investment are in relatively 

good condition and therefore require less investment. If so, the PJ should consider whether these are the 

most appropriate units to select for development assistance. How are more deteriorated units treated? 

Alternately, is there a problem with insufficient investment? (In other words, is the investment enough 

to bring the property up to code?) 

5 Homeowner rehabilitation costs are much greater than the state, and exceed the national average 

costs. Note, for cost data, the state data would typically be more meaningful. For this PJ, these questions 

should be answered: 

a. Is the cost data accurate? If not, is this problem related to personnel or procedures? 

b. If the cost data are accurate, are the high costs due to: 

• Contractor pool? For instance, if there are too few contractors participating in the program, there 

may not be sufficient competition to keep costs reasonable; 

• Is staff comparing actual bids to staff bid estimates prior to contractor selection? 

• Is PJ too generous in its rehabilitation standard or policy on eligible costs (in other words, does the PJ 

pay for materials, appliances, or construction elements that are more than modest?) 

• Is there problem with fraud or abuse? 

6 CHDO disbursement rate lags behind the overall program disbursement rate. This PJ is doing an 

outstanding job with committing and disbursing HOME funds in a timely manner. It is expected that the 

commitment and disbursement rates for the program overall would be comparable to the completion and 

disbursement rates for the rental program and the CHDO activities. For this PJ, the rental disbursements 

and commitments are nearly the same as the overall rates. However, the CHDO disbursements, while very 

good and above average for the state, are still somewhat behind the PJ’s overall program. This might be an 

issue worth exploring. 

19
As this example clearly illustrates, data that provide a “big picture” can assist program managers in identifying 

possible performance-related issues that require further exploration. The SNAPSHOT alone is not sufficient to 

determine if the PJ is in compliance with program rules, is producing as efficiently as possible, or is having the 

intended program impact to meet the community's needs. However, the SNAPSHOT data certainly can guide a 

program manager to ask necessary questions, gather additional information, and make his or her own judgments 

about the PJ’s needs to ensure continuous improvement in the program.

HOME program managers should review IDIS reports at least quarterly, if not more often. Again, any apparent 

problems identified by IDIS data should not be considered definitive, but rather flagged for follow-up and further 

examination. Taken together, a manager’s experience and instincts, combined with careful observations and 

periodic review of the basic IDIS program production data, should help focus tracking and measurement efforts. 

With one or two programs identified for measurement, it is time to create a map of the production process. The best 

way to do this is to look closely at, and document, the production process (also called “workflow”) for the 

program(s) under consideration. 

Identify Key Program Milestones of the Production Process

The production process is the set of activities that comprises the program. Mapping the production process and 

documenting it in writing ensures common understanding of how the system works (or should work). 

Every production process can be mapped in terms of key stages, or “milestones.” Typically, there are products 

generated at each stage in the process, referred to as “interim outputs”. The map of milestones, with identified 

interim outputs, forms the foundation for all measurement activities. This foundation facilitates communication 

about expectations related to performance, in terms of time, quality, and roles. In addition, it makes it easier to track 

the performance of staff, contractors, service providers, and developers. 

Identifying Milestones 

A production process is generally composed of seven to twelve clearly delineated steps that represent the 

culmination of major groups of activities necessary to conduct the program and achieve stated and desired results. 

(Refer to Figure 2.2 for sample milestones by HOME activity type.) How the stages of a process are defined 

depends on the level of detail one wishes to consider when gathering or analyzing data. Any stage or activity can 

always be broken down into smaller, more discrete steps. However, care should be taken not to create an 

unmanageable process. In other words, it is important to understand the full complexity of a task, but to be 

selective in choosing what to monitor and measure. 

“Natural” milestones occur at steps that mark the completion of critical stages in the process and at points of 

transition in the program. Transitions might occur in: 

• Key staff responsibility (such as from project manager to inspector when a project is under construction), 

• Status of client/participant (such as from applicant to approved client), or 

• Type of work to be performed (such as from a loan closing—the completion of the underwriting stage—to a 

project under construction). 

Initially, the milestones might seem over-simplified, but they form the basis for establishing goals and tracking 

progress. Some questions to consider when selecting milestones include: 

• Does each milestone mark the end of a definable stage and the beginning of another? 

• Are milestones associated with measurable outputs? 

• Is the list of milestones adequate? 

• Do staff members agree that these milestones are good measuring points for the program? 

20
Figure 2.2 

Sample Production Process Milestones by Activity 

Activity Sample Milestones 

Owner-occupied housing 

rehabilitation 

Any of the major stages 

indicated here could be 

broken down into smaller 

sets of activities. 

▪ First contact with 

client 

▪ Application 

submitted 

▪ Applicant approved 

and specifications 

requested 

▪ Specifications 

complete 

▪ Contractor selected 

▪ Order to proceed issued 

to contractor 

▪ Construction complete 

and final payment 

issued 

Homebuyer assistance 

programs 

Homebuyer programs follow 

many paths depending on 

the level of assistance and 

when in the home-buying 

process clients engage the 

program.

▪ Outreach and 

homebuyer fairs 

complete 

▪ First contact with 

client and preliminary 

credit review 

▪ Application submitted 

▪ Purchase contract 

executed 

▪ Inspections completed 

▪ Lead paint issues 

resolved 

▪ Loan approved 

▪ Purchase completed 

Rental development 

The milestones in this row 

represent the program’s 

perspective. 

▪ Community need 

analyzed and RFP 

issued 

▪ Proposals reviewed 

and selected 

▪ Design approved 

▪ Feasibility approved 

▪ Plans approved 

▪ Contract executed and 

order to proceed issued 

▪ Construction complete 

▪ Rent-up complete 

Rental development 

The milestones in this row 

are designed to measure 

the performance of a 

developer. 

▪ Development team 

identified and 

resumes submitted 

▪ Market analysis 

complete 

▪ Conceptual design 

complete and 

potential sites 

identified 

▪ Site selected and site 

control obtained 

▪ Engineering studies 

complete 

▪ Plans complete 

▪ Plans approved 

▪ Construction complete 

▪ Rent-up complete 

▪ Asset management and 

monitoring 

21
Figure 2.2 

Sample Production Process Milestones by Activity 

Activity Sample Milestones 

Tenant-based rental 

assistance (TBRA) 

The analysis of a TBRA 

program may start by 

looking at time issues, but is 

most likely to focus on data 

from outside the program, 

such as the number of 

affordable, suitable units 

available for rent. 

▪ Client accepted on 

waiting list 

▪ Application submitted 

▪ Application approved 

and voucher issued 

▪ Property identified 

▪ Inspections completed 

▪ Lease signed 

▪ Annual re-certification 

Maintaining the asset and 

monitoring compliance 

These are examples of 

milestones pertaining to 

assuring the quality of 

program assets and 

monitoring compliance with 

ongoing requirements.

▪

▪ Initial rent and 

occupancy review of 

rental projects 

▪ Review of project 

compliance reports for 

rental projects and onsite monitoring reviews 

(if applicable) 

▪ Physical inspections of 

rental projects 

▪ Regular monitoring of 

resale/recapture 

provisions for homebuyer 

projects 

▪ Regular monitoring 

reviews of TBRA program 

activities 

Interim Outputs

Once the milestones are identified, a list of the interim outputs associated with each step in the process can be 

generated. Interim outputs refer to the products that are produced throughout the production process that are 

necessary for completion of the final product. Examples of interim outputs include an executed loan agreement and 

approved construction plans or specifications. By identifying and measuring key interim outputs, a manager can 

determine where in the production process things are not working at optimum efficiency. 

Dimensions of Production: Structure, Time, Accuracy, and Quantity

Once the program milestones are identified, the production process can be reviewed and analyzed. There are four 

dimensions involved in managing, and therefore analyzing, a production process: structure, time, accuracy/quality, 

and quantity. Studying each dimension is necessary to gain a full understanding of an agency’s productivity. Each 

dimension is defined as follows: 

• Structure. This involves mapping out the sequence of events that a program entails in order to gain an 

understanding of the relationships among those events. 

• Time. This dimension refers to the length of time it takes to get through each stage of the process and, 

subsequently, the overall process itself. 

• Accuracy and quality of both interim and final outputs. By inspecting and comparing outputs, one can 

evaluate whether they are accurate or of adequate quality. 

• Quantity. Quantity refers to the number of interim and final outputs produced, perhaps by individual 

employees or by the overall program itself. 

22
A thorough analysis of the program’s productivity would include establishing performance measurements and 

analyzing performance for each of these dimensions. At this stage, however, it may be clear that one or two of these 

dimensions are more problematic than others. Having completed the process of reviewing programs from a big 

picture perspective, and having mapped out the production process, a housing program manager should be able to 

make a determination about which of the four dimensions should be analyzed first. 

Step 2. Identify Needed Data and Determine Potential Data Sources 

At this point, the manager should have a good understanding of his or her programs, and should have identified one 

program, or perhaps one specific dimension of a program, that needs to be examined and improved. Next, it is time 

to determine what data will be needed to better understand what is occurring within that program. 

Depending on the type of problem that has been identified, the cause(s) of the problem may or may not be apparent 

at this stage. For instance, it may be that the problem is initially defined only in general terms—such as, the 

expenditure rate for the homebuyer assistance program is not sufficient and the PJ is at risk of losing HOME funds 

for not meeting the five-year regulatory requirement. Alternatively, it may be very apparent that the homebuyer 

assistance program attracts a limited number of qualified applicants, and therefore is unable to meet its goals. The 

more a manager knows about the causes of any problem in the program under review, the more he or she will be 

able to target data needs. 

It is not necessary that the causes of the problem be well understood at this phase of the process. In fact, it is 

sometimes useful to “test” assumptions about the causes of the problem(s) by generating a number of possible 

causes and collecting data to determine which of the hypotheses are valid. In reality, a program manager may find 

that there are a number of issues contributing to the problem-not just one thing. Further, as data is collected, it is 

common to re-evaluate and redefine the nature of the problem. 

To determine what data must be collected, and to identify potential sources of that data, a number of related tasks 

must be undertaken: 

• Hypothesize about the possible causes of the problem(s) identified in order to be sure that all possible 

causes are at least considered. As data is collected, it is common to re-evaluate hypotheses and subsequently 

redefine the nature of the problem. Accurate diagnosis of the problem is critical to effective problem 

solving! 

• Develop performance targets for the program in order to have a benchmark against which to measure actual 

performance. 

• Identify what data is needed to determine if the hypothesized cause has validity and if the targets are being 

met. 

• Identify potential data sources.

• Evaluate the proposed data sources, selecting those that will provide the most reliable data at the least 

expense. 

Hypothesizing about causes

Hypothesizing about the possible causes or source of the problem(s) facilitates the decisions concerning what data 

should be collected and analyzed. The list of possible causes should be as exhaustive as possible, and every 

possible explanation for the problem should be generated. Staff, clients, contractors, and subrecipients can provide 

valuable input in this process. Complaints from (or about) any of these parties are also important sources of 

information. 

Generally speaking, program performance problems can be traced back to one, or some combination of, the 

following categories: 

• Program design issues, such as who is being served, type and level of assistance, and length of assistance. 

Note that problems with program design often lead to an inability to meet program outcome goals, as 

discussed in Chapter 3. 

• Personnel/staffing issues that relate primarily to staff performance, training, motivation, and staff workloads. 

23
• Processes, procedures, and/or documents, related to the number and order of steps involved in completing 

a task, including the amount and timing of staff oversight. Inadequate documents and written agreements can 

also cause problems. 

Figure 2.3 provides an analysis of some common productivity problems and how they might be interpreted relative 

to these three programmatic aspects. 

Figure 2.3 

Sample Productivity Problems and Hypothesized Causes 

Problem Program design Personnel and 

staffing 

Process, procedures

Failure to spend funds 

for homebuyer 

assistance program in 

timely manner. 

Program carries a 

cap on assistance 

that is insufficient. 

Income-eligible 

buyers are not able 

to complete a 

purchase without 

more financial help. 

Program 

marketing 

efforts have 

been 

inadequate due 

to understaffing. 

Application process 

is cumbersome for 

the buyer. 

Steps are 

duplicative and timeconsuming. 

PJ staff spends a 

considerable amount of 

time resolving 

homeowner complaints 

about contractors in 

homeowner rehabilitation 

program. 

Homeowner is not 

involved in key 

decisions concerning 

what can be 

renovated in the 

home and what 

cannot. 

Contractor does 

substandard 

work. 

PJ and contractor 

assume the other 

party is coordinating 

with the owner and 

providing notice of 

work dates, etc. 

Development projects 

are not committed in a 

timely manner; loan 

approval process is 

extremely long. 

Program does not 

have established 

program priorities, 

nor clear guidelines 

for staff and 

developers. 

Staff 

responsible for 

underwriting 

and loan 

approval is not 

adequately 

trained, and is 

not clear on 

responsibilities. 

Underwriting 

process is 

cumbersome and 

does not include a 

screening 

mechanism through 

which poor project 

proposals can be 

identified and 

rejected early in the 

process. 

Inspections reveal a 

significant number of 

jobs that are not 

satisfactory. 

Rehabilitation 

standard needs to be 

revisited, updated, or 

revised. 

Contractors 

need additional 

trainings in 

some areas of 

federal program 

construction. 

In-progress 

inspections are 

either not timed 

appropriately, or are 

not being conducted 

at all. 

Develop Program Targets 

Based on informed hypotheses about the nature of the problem and the factors that may be contributing to the 

problem, program targets should be identified for milestones and variables that have the greatest impact on 

production. Establishing targets in and of itself can improve productivity because it provides a marker for which 

staff, contractors, and other program partners can strive. 

24
Targets might relate to any of the four dimensions (structure, time, quality or accuracy, and/or quantity), but they 

should always be specific. Consider the following examples: 

• The cost of change orders on the homeowner rehabilitation program should be 10 percent of total 

construction costs; 

• The total number of rehabilitated rental units should be 75; 

• Punch list items on all rehabilitation jobs should be completed within 30 days of the issuance of the 

certificate of occupancy; or 

• Staff should complete a review of a development proposal within 30 days of receipt. 

Targets will be hard to establish and project without some data to define where the agency is at the outset of the 

process. In the early stage of productivity measurement, the most important step may be simply collecting that 

baseline data. 

Data related to the timeliness of performance is especially useful for planning and for evaluating program progress. 

With it, one can answer the following types of questions: 

• How long does it take to complete market studies? How long should it take? 

• How long does it take for rehabilitation inspectors to produce a good set of specifications? 

• How many cases can one loan underwriter work on at the same time? 

• How long does the permitting process take? How long should it take? 

Using the milestones that were identified in the documentation of the production process, managers can work with 

staff, contractors, and subrecipients to identify “time goals” for each stage. Time projections must be realistic and 

based both on past experience and negotiations. Involving staff and program partners to establish practical time 

targets can be a powerful tool in ensuring good performance and identifying any problems early on. Time goals 

provide a clear understanding of expectations and also provide an objective basis by which to measure performance. 

With time goals in hand, particularly when established in conjunction with program partners, parallel reporting 

systems and time-based performance measures should be established. These can form the basis of a “price of entry” 

system for continued funding for subrecipients and contractors. For instance, one of the largest productivity 

problems facing programs today is the failure of developers who have been awarded funds to move forward and 

complete projects. One way to address this problem is through a time-based performance management system. A 

chart similar to Figure 2.4 below, accompanied by noncompliance or enforcement procedures, could be included as 

part of any award of funds for development. 

Figure 2.4 

Rental Development Milestones

Rental Development Milestone Allowed Elapsed Time From Contract 

Execution 

Development team identified and resumes submitted 45 days 

Market analysis complete 60 days 

Conceptual design complete and potential sites identified 120 days 

Site selected and site control obtained 180 days 

Engineering studies complete 210 days 

Plans complete 240 days 

Plans approved 270 days 

Construction complete 450 days 

Occupancy complete 750 days 

Asset management and monitoring Ongoing 

25
In a time-based performance management system for this rental development, developers would be required to 

submit monthly reports that specifically address their progress in meeting each of the identified milestones. 

Developers would be asked to explain in detail the reason for not meeting any milestone, and program staff would 

monitor the developer and provide technical assistance and/or enforce penalties if the progress was not timely. 

A similar system could be developed for a state recipient or subrecipient administering a HOME-funded activity. 

The PJ could work with its housing partners to develop milestones that the partner is responsible for meeting. The 

housing partners would report on progress towards the specific milestones on a regular basis, and would be required 

to provide an explanation whenever progress is stalled. With such a system, the PJ is well-informed concerning 

what types of technical assistance might be needed. For partners who consistently have difficulty meeting 

milestones, other interventions might be necessary. Chapter 4 presents a case study illustrating these principles. 

Identifying Data to Collect 

With several assumptions about the nature of the issue to be addressed, the next step is to determine what data will 

be needed to determine if the assumptions are correct. It is important to carefully think through what information is 

needed and how it will be used after it is collected. Making 

these determinations should be a joint decision among 

management and staff, and may involve other program 

partners as well. 

For each possible problem that is identified in step one, a 

companion list should be generated that describes what type 

of data might help determine the cause of this problem. This 

list should be as specific as possible, although its specificity 

will be driven, in large part, by the nature of the problem 

under review. Figure 2.5 provides an example of how the 

list might look. 

Before beginning data collection, data must be clearly 

defined, and staff must be trained to ensure data is collected in a consistent manner. If more than one person is 

collecting data, it must be collected consistently and all staff must define things the same manner. For example, if 

an agency is measuring “completed” units, it is critical that all involved staff have a common definition of 

“completed.” Is a unit complete when construction is complete, or must the unit be occupied? 

Data collection takes time and 

costs money. Plan carefully so 

that you collect only that 

information you need and can 

use to understand or solve your 

problem(s)! 

26
Figure 2.5 

Identifying Data Needs 

Problem: IDIS data shows that the PJ has failed to spend funds for its homebuyer assistance 

program in timely manner. 

Possible causes of problem Data Needs 

Program design 

The program carries a cap on assistance that is 

insufficient. Income-eligible buyers are not able 

to complete a purchase without additional 

financial aid. 

Determine why applicants dropout of the process. 

Evaluate applicant financial 

information to determine whether 

sufficient assistance is offered. 

Compare financial information of 

approved, rejected, and drop-out 

applicants. 

Personnel and staffing 

Program marketing efforts have been 

inadequate. 

Identify use of staff time and 

workload. 

Determine if marketing efforts 

reached potential pool of clients. 

Process, procedures 

Process is too cumbersome and time 

consuming. 

Track each milestone of the 

process by time. Determine how 

many applicants drop out and at 

what stages. 

Determine level of customer 

satisfaction with application 

process (of approved, rejected 

and dropped out). 

Identifying Data Sources

Agency files provide a readily available source of data. Managers may be surprised at the amount of information 

available at their fingertips—a significant amount of information can be collected simply by extracting the data from 

individual files and compiling it into a format that allows for easy analysis (such as a spreadsheet or database). 

Following are some examples of the types of data that can be extracted from agency files: 

• Project files that have a chronology sheet attached to the cover (as they should), provides time data for the 

program stages. 

• Construction files provide information about the bidding process, including cost estimates and actual bids (for 

purpose of comparison), and number and dollar amount of change orders. 

27
• Dates on application forms and approval documents provide data about elapsed time (such as, average length 

of time applicants sit on waiting lists). It also provides information about the demographics of populations 

served and not served. 

While agency files provide important information on program activities and applicants, they likely will not provide 

all of the information needed. Agency forms, applications, and program procedures can sometimes be modified to 

collect additional information. Other times, it is necessary to turn to external data sources. Appendix I lists 

common national housing and community development data sources. 

PJs may also determine that they need new methods to collect data directly from program staff, partners, clients, and 

applicants. Chapter 3 provides guidance on other methods for data collection, including surveys, focus groups, and 

interviews. Regardless of the method, however, it is important that staff be trained on how data elements will be 

defined and collected to ensure that the data collected among various staff is comparable. 

Evaluate the Sources of Data and Determine Availability of Data

Once the universe of possible data sources has been identified, each source must be evaluated in terms of its 

usefulness in the performance measurement process. For some types of information, there may be more than one 

data source available. 

For each possible data source, the following questions should be addressed: 

• Will the data measure what it is intended to measure (especially if outcome indicators are used)? 

• How reliable will the data be (in other words, is very specific information available, or only general proxies)? 

• Is the data already collected or can it be easily collected? If not, what will data collection entail? (That is, 

how much time and staff effort would it involve, and how much would it cost?) 

The most important question is whether or not the data adequately measures what it is intended to measure. Data 

collection efforts are meaningless when the source is not adequate or appropriate. Figure 2.6 identifies some 

common questions a PJ might want to evaluate, and where the data might be found to undertake that analysis. 

Figure 2.6 

Where to Locate Data to Answer Common Questions 

If a PJ wants to 

know... 

Data Source Data Needed 

If staff cost estimates are 

in an acceptable range Construction files 

Cost estimate vs. bids vs. 

bid award 

If applicants drop out of 

the process because the 

subsidy is insufficient 

Application Compare income of 

approved, rejected, and 

dropped out applicants 

Number of units 

produced per FTE staff 

PR-22 (Status of HOME 

Activities report) and 

Time sheets 

Ratio of units completed 

per FTE 

Average length of time to 

complete a unit 

Application form and 

Construction files 

Average time lapse from 

initial contact to 

construction completion 

(might use date of 

certificate of occupancy, or 

date of actual occupancy 

(lease, deed) 

If change orders are 

excessive 

Construction files Average percentages of 

cost of change orders / 

total cost of construction 

28
Step 3: Apply Measurement Tools 

Once it is clear what data is required, the third step in the process is to begin collecting data. As mentioned above, 

data collection tools should be created so that staff can be compiling data during the course of their day-to-day 

work, rather than opening all program files on a periodic basis to collect this information at a later date. 

Three powerful tools can accomplish most managers’ needs: 

• Flowcharts; 

• Spreadsheets; and 

• Individual Performance Reports. 

The use of each of these tools requires varying levels of 

sophistication, and each has appropriate uses. Plan in advance to 

determine the best tool for the job by asking: 

• Will this tool enable the agency (or its subrecipients) to 

improve productivity or help solve a problem? 

• Can this tool be designed so that it fits the program being measured? 

• Does this tool allow data to be collected and compiled in a way that does not create an undue burden on 

program partners or staff? 

Figure 2.7 provides some examples of the types of tools that work best for a sample of common productivity 

problems. 

Figure 2.7 

Addressing Productivity Problems 

Problem or Question Process and Tools 

❑ Are we committing and spending funds 

according to HUD timeliness requirements? 

❑ How many final outputs are we producing 

per year or month? 

❑ IDIS reports and simple mathematical 

projection 

❑ Are we experiencing undue delays in 

projects, imbalances in workload, or a large 

number of complaints? 

❑ Observation and interviews with staff. 

❑ Establish milestones and monitor 

performance against them. 

❑ Are we producing enough at each stage of 

the production process to assure that we 

reach our goals and balance our workload 

over the year? 

❑ Set milestones and monitor 

performance. 

❑ Use spreadsheet tracking tool. 

❑ Is there a rational organization of steps and 

responsibilities in our production process? 

❑ Simple flowchart 

❑ Are CHDOs, other subrecipients, and 

employees achieving a reasonable number 

of outputs. 

❑ Are outputs of acceptable quality? 

❑ Spreadsheet tracking and 

spreadsheet analysis of outputs 

❑ How is each employee spending his or her 

time? Is each employee carrying out the 

right tasks, structuring his or her time 

effectively, and producing the optimum level 

of outputs? 

❑ Spreadsheet analysis, coupled with 

individual employee performance 

reports and interviews 

Remember... 

It is critical that data 

collection efforts be 

integrated into normal, 

everyday operations of the 

staff. 

29
Using the Flowchart to Address Structural Issues 

Flowcharts are commonly used to examine all of the individual activities within a distinct stage of a production 

process. A flowchart elaborates considerably on the production process milestones by diagramming the process in 

far greater detail. Typically, it also identifies the individual responsible for each step. Charting this information 

simplifies the evaluation of the relationship between tasks and the relationships between staff members. It can be 

helpful for: 

• Eliminating duplicative efforts; 

• Identifying sequential tasks that can be performed simultaneously; 

• Ensuring that the right person is doing each job; 

• Determining whether or not workloads are balanced; and 

• Assessing whether or not some work might best be subcontracted. 

Unlike other tools that need to be used consistently in order to be useful, flowcharts are used only when designing 

or redesigning a production process or investigating a structural production problem. 

30
Rock City Specification Writing Process

Conduct Thorough

Inspection

Verify and apply 

applicable codes 

and standards

Complete Preliminary

Estimate

Prepare Preliminary

Scope

Write specifications

Confirm prices

Prepare final estimate

Homeowner

Desires

Revise Preliminary

Scope

Alternative solutions,

methods, or materials

Recheck prices

Meets

Budget

Over

Budget

Analyze problems

Verify code requirements

Secure alternate solutions

Select materials

Complete schematic drawings

Determine construction schedule

Secure outside

technical assistance

Meets

Budget

Exceeds

Budget

To

Homeowner

Figure 2.8 

Flowchart for Activities Within Single Stage of Production Process 

Using Spreadsheets to Measure Time, Quantity, and Accuracy

Spreadsheets are used for daily or weekly recording and processing of numeric data, allowing one to track the 

quantity, timeliness, and accuracy of outputs. Figure 2.9 is a sample productivity report that can be maintained on a 

spreadsheet. Spreadsheets can be used to track and tabulate a large quantity of information and can help managers 

to: 

• Understand how long it takes to produce a product from start to finish. This information will help determine 

whether there are enough jobs in each stage of the pipeline to keep all job functions busy. 

• Keep track of a large number of projects simultaneously. 

• Identify the stage(s) in which projects are delayed. 

31
• Measure the quantity and accuracy of outputs produced by individual employees and evaluate whether 

workloads are distributed appropriately across job functions. 

• Tabulate data to find averages, ranges, and variations between data variables, such as costs and change 

orders. 

• Measure the accuracy and quality of work, such as: 

— Construction specifications (by tracking the range of bids received and the number of change orders); 

— Cost estimates (by comparing estimates with bid prices and final prices); or 

— Construction work (by tracking complaints or warranty call-backs). 

Spreadsheets are one of the most practical tools available for tracking data over time. Staff and partner 

organizations can collect data during the course of their normal activities and enter it directly into a spreadsheet. 

This data can be aggregated and manipulated in many useful ways for analysis. Note that some database programs 

perform similar functions, but typically, data manipulation needs can be served sufficiently by a spreadsheet. 

As with most data tools, spreadsheets are a tool for collecting and tabulating data to help analyze problems (or 

potential problems). However, spreadsheets do not diagnose the cause of problems. For instance, the data might 

identify a “log jam” in program operations, but managers and staff must identify the cause(s) of the log jam. The 

cause(s) could be related to a number of things: staff member(s) may have more work than can be done by one 

person; an uncontrollable external factor is impeding the work; or more training is needed. Prudent managers look 

for the story behind the data, and use the data to target questions, inform discussions, guide investigations, and 

make decisions. 

 

32
Figure 2.9 

Project Tracking Spreadsheet – Rock City Owner Occupied Rehabilitation Program 

Project #

First 

Contact 

Days 

Elapsed

Application 

Submitted Staff

Days 

Elapsed

Approved, 

Specs 

Requested Staff

Days 

Elapsed

Specs 

complete

Days 

Elapsed

Contractor 

selected

Days 

Elapsed

Contr

execu

29.2 39.6 9.1 39.4 17.2 

4 12 3 23 9 

Max. Days Elapsed 63 67 19 45 31 

2 00 3 1 

13% 0% 0% 30% 14% 

1 5/4/2000 61 7/4/2000 CJ 39 8/12/2000 AK 4 8/16/2000 37 9/22/2000 24 10/16/22 5/7/2000 63 7/9/2000 CJ 34 8/12/2000 MO 4 8/16/2000 Dropped d N/A 

3 5/10/2000 58 7/7/2000 DP 37 8/13/2000 MO 3 8/16/2000 44 9/29/2000 14 10/13/2

4 5/11/2000 Dropped d CJ N/A N/A N/A N/A 

5 5/30/2000 12 6/11/2000 CJ 67 8/17/2000 AK 19 9/5/2000 23 9/28/2000 31 10/29/26 5/30/2000 Dropped d EM N/A N/A N/A N/A 

7 6/16/2000 31 7/17/2000 EM 39 8/25/2000 AK 11 9/5/2000 44 10/19/2000 15 11/3/28 6/16/2000 52 8/7/2000 DP 36 9/12/2000 MO 3 9/15/2000 Dropped d N/A 

9 6/22/2000 29 7/21/2000 CJ 46 9/5/2000 AK 

10 7/23/2000 0 7/23/2000 DP 40 9/1/2000 MO 13 9/14/2000 Dropped d N/A 

11 7/25/2000 4 7/29/2000 CJ 48 9/15/2000 AK 14 9/29/2000 45 11/13/2000 10 11/23/212 8/1/2000 6 8/7/2000 EM 38 9/14/2000 MO 5 9/19/2000 39 10/28/2000 Dropped d 13 9/8/2000 20 9/28/2000 CJ 14 9/14/2000 14 9/28/2000 DP 12 10/10/2000 MO 15 10/25/2000 44 12/8/2000 9 12/17/215 9/15/2000 CJ 

Dropout Rate# DropoutsMin. Days ElapsedAvg. Days Elapsed

33

Individual Performance Reports

An individual performance report can help managers better understand the work of each staff person, and thereby 

make more informed decisions about work assignments, performance assessments, and needed training. 

With regular reports on individual performance, managers are better able to recognize individual success and 

identify capacity problems in terms of an individual’s skills. The latter can help a manager see the need for 

technical assistance and training for staff, or a shift in job functions to others. Alternately, celebrating individual 

success can motivate staff to produce good results. Individual measurement provides opportunities for each staff 

person to understand how his or her own work has a positive impact on the mission of the organization, and 

ultimately in the community. Effective managers should use this information to recognize good work, reward it 

when possible, and collectively celebrate successes. 

To design an individual performance report, the manager should establish a format for each individual staff member 

to keep a log of critical work activities. The report should be completed each month. The scope of the report is 

limited to key elements of each staff person’s job, which may change over time as certain issues become more or 

less critical in the performance of that job. 

For example, an individual performance report for a homebuyer assistance program specialist might include the 

following data for the month: 

• Number of outreach events, such as homebuyer fairs, planned and completed; 

• Number of applications processed; 

• Number of rejections/referrals for bad credit; 

• Number of inspections completed for HOME and lead 

compliance; 

• Number of loans approved; 

• Number of properties in progress waiting for repairs; 

• Number loan applications in progress waiting for bank 

approval to close (and/or other reasons); 

• Training attended; and 

• Other job duties or special projects carried out. 

Tracking staff activities that are not tracked for the program 

can uncover activities or external forces that dramatically 

impact staff’s time. This can lead to discussions on how to change the way the agency does business and to 

increased productivity. 

Conduct a Trial Run 

A simple pilot effort to test data collection tools and 

procedures can save enormous time and energy by 

flagging a number of common problems early in the 

process, including: 

• Cumbersome data collection procedures; 

• Data quality control issues; 

• Overlooked outcomes; 

• Inadequately defined indicators; 

• Data entry procedures; and 

• Data analysis/interpretation problems. 

Step 4. Analyze the Data 

Analyzing the data is the most critical step in the productivity measurement process—figuring out exactly what the 

data means. 

The data analysis process can be broken down into several components: 

• Process data and check for errors 

• Tabulate the data 

• Analyze the data and interpret the results 

35
Data Entry 

Once data is collected, it may need to be transferred to a format to facilitate data manipulation and review. This 

function is referred to as “data entry.” Data entry is required when the raw data is collected in a format that is not 

conducive to analysis, such as on individual forms, in files, or on questionnaires. The data is typically transferred 

into some type of computer software system or on a new form that makes tabulation easier. A number of easy-touse software packages (for spreadsheets or databases) are available. 

Data entry is simplified when it is done on a regular basis, rather than all at once just prior to analysis. Regardless 

of when it is done, it is important that staff understands how to enter various types of data, particularly if data is 

qualitative in nature and must be coded, or when the data is left blank (e.g., if a respondent did not answer a 

question on a survey). Prior to analysis, it is also important to check for errors in the data entry process. 

Tabulate the Data 

Data tabulation involves generating basic statistics related to the findings. Depending on the type of program, this 

might include: 

• Total number of units; 

• Averages or median income, purchase price, or rents; 

• Cost per unit, per Full Time Equivalent (FTE); 

• Average change in property values; and 

• Average number of days required to process an application, complete construction. 

Analyze the Data 

Correctly identifying the factors affecting a program’s performance is imperative to identifying meaningful 

solutions. The data is the starting point. 

Benchmarks are commonly used to facilitate data analysis. A benchmark is a standard by which one measures 

progress. Some commonly used benchmarks include: 

• The PJ’s own projected targets (See Step 2 above); 

• Industry standards, 

• Performance of a similar jurisdiction, if available; and 

• The PJ’s own performance, as measured over time on a periodic basis (such as quarterly). 

When reviewing program data, managers should look for: 

• Any unexplained or dramatic changes in the data over time; 

• Differences between benchmarks and actual data; and 

• Patterns in the data that seem to be related to particular staff, neighborhoods, contractors/developers, 

landlords or property owners, or other key players (for instance, data might show certain developers are 

consistently behind schedule, while others are not). 

After examining the data as a whole, it is often useful to break down the data and compare the results by category, 

such as race, age, household size, income level, educational level, neighborhood or census tract, etc. Demographic 

breakdowns are particularly valuable when trying to evaluate who or what neighborhoods are being served (and 

who is not). For example, comparing jurisdiction-wide homeownership rates by race or census tract with program 

participation rates may provide valuable insights about the effectiveness of program outreach efforts. 

Encouraging staff and program partners to help analyze the data is a way to cultivate “buy-in” and encourage 

problem-solving from those most directly in control of the work. In fact, data analysis may raise more questions 

than it answers. Involving others in the analysis and eliciting additional qualitative information from these sources 

may become important. Qualitative information helps explain the story behind the numbers so that problems and 

issues can be solved. Discussions with staff or program partners are an absolute must in the process of diagnosing 

the “why” behind the data and understanding the true barriers to productivity in programs.

36
Chapter 3: 

 Measuring Program Outcomes 

Chapter 2 explains how measuring productivity helps a program manager determine how much an agency 

or program has produced. Chapter 3 explores how measuring outcomes can help a manager monitor the 

impact the agency’s program is having on the individuals or community it is designed to serve. 

Monitoring program outcomes can guide agency improvement efforts and ensure that changes are made to 

programs that are not achieving the intended outcomes. 

What is a Program Outcome? 

Program outcome measurement is focused on measuring whether or not a program is meeting its intended purpose. 

The program outcome differs from a program goal in that a goal is a statement of the accomplishment the agency 

plans to achieve—it is frequently articulated as some unit of production, such as a housing unit or a household 

assisted. The outcome is the change one wants to make. For instance, the goal of a homeownership rehabilitation 

program may be the rehabilitation of fifteen units of substandard housing. The intended outcomes may be to 

revitalize a neighborhood or to make it safer. As this example suggests, program outcomes can be long-term goals, 

and in some instances, may take many years to realize. 

What are the Benefits of Measuring Program Outcome? 

The purpose of housing programs is often tied to other community development objectives as well. There is a 

growing recognition of the need to integrate HOME funds into comprehensive approaches to solve community 

problems. These include neighborhood revitalization programs, community-wide workforce development/economic 

development strategies, and homeless strategies. Because of their multi-faceted nature, however, it is essential that 

comprehensive strategies include an assessment of each component in order to determine where strategies have been 

effective and where adjustments are needed. 

For many organizations, monitoring results rather than processes is a new way of thinking about success. However, 

there are three basic reasons why outcome measurement is essential to success: 

• Outcome measurement supports long-term planning. Significant changes in a community are not likely 

to happen overnight. As a result, it is important to review, adjust, and improve programs and projects along 

the way. By defining key milestones used to judge progress, outcome measurement helps to keep long-term 

plans on track and ensure that resources are targeted for the greatest impact. 

• Outcome measurement facilitates mid-course corrections. Changes in the regional and national economy, 

migration into and out of the community, election of a new mayor—these are only a few of the factors that 

can affect how well programs and projects perform. Outcome measurement creates an early warning system 

that shows where shifts in the larger political, social, and economic environment are impacting a 

community’s progress and helps the community make mid-course corrections to solve problems and 

capitalize on new opportunities. 

• Outcome measurement allows effective use of scarce resources. While HOME funds are significant, they 

are not infinite. Through evaluation of the results created by programs and projects, performance 

measurement can help channel resources where they make the most difference. With the right type of 

information, changes can be made to keep housing strategies responsive to changing community needs. 

37
The Process of Measuring Program Outcomes 

The process used to measure program outcomes is comparable to the process used to measure productivity, 

described in Chapter 2. This process can be broken down into the same six steps: 

1. Identify/confirm program goals and intended outcomes; 

2. Identify the data needed and determine data sources; 

3. Select and apply measurement tools; 

4. Analyze the data; 

5. Implement changes and improvements; and 

6. Continue measuring program performance. 

While this publication describes the measurement process for productivity and program outcomes separately (in 

Chapters 2 and 3, respectively), measuring productivity and measuring outcomes are not two separate processes and 

typically are not addressed individually. An effective performance measurement system will address components of 

both, particularly since productivity can have a significant effect on program outcomes. These relationships will be 

explored further in Chapter 4. 

Evaluating program outcomes involves the following steps and issues: 

Step 1. Identify/Confirm Program Goals and Intended Outcomes 

For agencies undertaking an outcome measurement process for the first time, it is advisable to start with one 

particular program with defined program outcomes. Before any assessment of current status can begin, the intended 

program outcomes must be defined. However, this is not always as easy as it sounds! As a result, it should be the 

subject of wide ranging discussions, both with staff and other stakeholders of the program. There must be some 

level of agreement on intended outcome(s) prior to any effort to measure whether those outcomes are being 

achieved. Staff and other stakeholders often have varying views of what a program is trying to achieve. The 

process of asking all stakeholders to focus on and come to agreement about program purpose can increase service 

effectiveness before data collection even begins. 

During the course of these discussions, it will likely become apparent that in addition to the specific program 

activities under review, there are many other factors that affect a particular outcome, particularly when the desired 

benefits are to an entire neighborhood or community. Many of these factors will be outside the agency’s control, 

and housing resources alone are typically not enough to effect dramatic change. When the desired outcomes are 

comprehensive in nature, PJs should consider involving other agencies (such as the police department or human 

services department) in their discussions about outcomes in order to collaborate, share information, and better target 

services to maximize program impact. 

Identifying Outcome Indicators 

Defining a program’s intended outcomes can be difficult, but the bigger challenge can be determining how to 

measure those outcomes. Some outcomes are fairly easy to observe and measure, such as an increase in the 

homeownership rate for a specific population or in a certain neighborhood. Many outcomes, however, are not able 

to be measured directly, and perceptions are more difficult to measure than hard facts. For example, there is no 

universally accepted threshold that defines when a neighborhood has been revitalized or when it can be considered 

safe. 

To measure such outcomes, one must identify indicators to represent the desired impact. An outcome indicator 

requires two things: 

1. A specific observable, measurable characteristic or change that will represent achievement of the outcome, 

and 

2. A specific statistic used to summarize the level of achievement, also known as a performance target. 

38
At the beginning of the measurement process there will be little basis for choosing performance targets. (Is the goal 

to increase property values by 10 percent or 40 percent?) Initially, it might suffice to aim simply for improvement 

in the selected indicators. The program can then establish numerical performance targets later, after a round or two 

of outcome measurement reveals what numbers are reasonable. 

Outcomes can be multi-dimensional, and one indicator may not capture all aspects of an outcome. As a result, some 

outcomes may require more than one indicator. For example, to measure a neighborhood revitalization effort, 

several indicators may be used: 

• The vacancy rate in the neighborhood; 

• Increases or decreases in property taxes; 

• Increases or decreases in average home sale prices; 

• Increases or decreases in neighborhood safety; 

• The homeownership rate; and 

• The level of private investment in the neighborhood. 

Step 2. Identify Data and Data Sources Needed to Measure Chosen Outcome(s) 

Unlike productivity, which is typically related to an agency’s internal operations, outcomes are generally related to 

the community-at- large (or the well-being of assisted households). Therefore, it is logical that at least some of the 

outcome data that an agency needs will be generated outside the agency. Depending on the specific outcome, 

however, data sources will vary. There are many data sources to consider: 

• Primary sources. Primary source data includes data that is generated by program activities. It can be 

extracted from agency records, and/or collected directly from applicants, program participants, and other 

program stakeholders (through surveys, focus groups, windshield surveys, etc.). 

Before undertaking new data collection activities, it is prudent to identify the types of data already collected by 

examining the forms and records used at each stage within the program’s process. Most agencies have a significant 

amount of information available in participant files, applications, and other records. 

• Secondary sources. Secondary source data includes data that is collected and compiled either nationally or 

locally. It is important to consider the use of secondary sources in outcome measurement because they are 

independent, typically reliable, and often standardized. Some of the most common secondary sources for 

housing and community development data include: 

— Data compiled by the local planning department; 

— Census data, including the American Community Survey Data, American Housing Survey Metropolitan 

Samples, and the Census 2000 Supplementary Survey; and 

— U.S. Department of Housing and Urban Development data, including A Picture of Subsidized 

Households, HUD Income Limits data, Fair Market Rents data, and the State of the Cities Database. 

Appendix I includes a list of secondary housing and community development data sources to consider. 

Keep in mind that there are some disadvantages to using in-house data exclusively. The validity of the data depends 

on how carefully it was collected and recorded. Furthermore, existing records seldom contain all the data needed, 

and caution has to be taken to ensure the data is current. And, unless the agency’s procedures already include 

customer service and satisfaction follow-up information, the data may need to be augmented using one or more of 

the tools discussed later in this chapter. 

39
Data Found in Program Records 

PJs may be able to obtain program outcome data from the following types of records. If the necessary data is not 

already collected, relevant data elements can be incorporated into the agency’s existing forms. 

• Approved and rejected applications and waiting lists. These records identify who is currently being 

served, who is about to be served, and who is not being served. Analyzing this type of data may lead to 

changes in the focus of marketing efforts or to the design of programs. 

• Siting decisions. Data related to siting decisions might assist in evaluations of whether the neighborhood 

or property targeting goals of the program are being met. 

• Unit designs. Analysis of data related to unit designs can help answer questions such as whether or not 

the agency is successfully developing units that are compatible with the neighborhood. In addition to inhouse information about unit design, holding neighborhood meetings can help staff assess whether units 

are well-liked by community residents and are meeting community needs. 

• Plans and specifications. Evaluating plans and specification will determine if standards and 

specifications are established with quality, durability, and energy efficiency in mind. Post-construction 

inspections will ensure that the work is completed according to the plans. 

Primary source data may also include impressions and perceptions from those involved in the program, such as 

participants, developers, and other stakeholders. This data is particularly important when the measured outcomes 

relate to changes in attitudes, opinions, or beliefs (for instance, whether or not residents of a particular 

neighborhood feel safer). In order to collect or capture this data, an agency may need to conduct surveys, focus 

groups, or interviews, which are discussed later in this chapter. 

To be valid measures of program effectiveness, outcome indicators must have measurability, accessibility, and 

consistency: 

• Measurability. The indicator should be objectively measurable, meaning not only should one be able to 

track change precisely, but different people doing the same measurement should end up with the same result. 

• Accessibility. The data selected should be generally accessible to the public and/or the agency. If securing 

the data is too difficult or cumbersome, data collection efforts will not last. 

• Consistency. For data comparisons to be useful, the data must represent the same thing from one year to the 

next, to ensure a comparison of “apples to apples”. For example, if tax assessments are used as an indicator 

of property values, it is important to verify that the tax code has not changed, thereby raising or lowering the 

base rate. 

Data accessibility is key to successful performance measurement. In many 

agencies, staff is fully utilized in program implementation. If data collection 

and compilation is too time-consuming, staff enthusiasm will dwindle and 

data collection efforts will not last. 

Collecting data to determine program impact is typically done on a periodic, rather than ongoing basis. Since most 

outcomes involve long-term change (such as changing lives and changing communities), it takes time for 

incremental changes to be measured. Determining the intervals at which to take these measurements should be a 

joint decision with program staff and other stakeholders. For many communities, it makes sense to tie this data 

40
collection and analysis effort to the annual planning process. After all, the analysis of whether a program is 

effective typically drives program planning efforts. 

41

Measuring Increased Housing Affordability

For many housing programs, the most significant intended outcome is to increase the affordability of housing for low-income 

households. The most common indicator of affordability is the percentage of income being spent on housing. When 

calculating housing costs, it is important to include utilities and/or other housing related expenses-such as condominium fees,

property taxes, maintenance, and insurance-to reflect the full cost of housing. 

Determining affordability can be tricky because of its definition: housing is generally considered affordable if a household’s

monthly cost does not exceed 30 percent of household income. However, there are some circumstances when this definition 

is neither relevant nor feasible, such as when household income is very low or fluctuates significantly from year to year or 

month to month, or when the cost of housing is extremely high and decent housing is scarce. Because of these issues, each 

locality must consider the issue of affordability carefully and develop a standard definition that make sense in its own 

community. 

Why Use This Indicator? 

The affordability ratio reflects two things: (1) How well a family fares in juggling housing costs versus other expenses; and 

(2) The success of an organization in securing affordable housing for rent or homeownership in relation to the local housing 

market and general economy. 

How Do I Get the Information I Need? 

First, primary stakeholders should discuss (and come to agreement on) what should be considered “affordable” in the context 

of the local market. Then, a formula can be developed to calculate affordability based on the determination of the maximum 

percent of household income that should be spent on housing costs in the local housing market. 

Next, the PJ should review its application form to ensure that it gathers needed baseline data, such as household income and 

current housing costs. The PJ can use an annual survey of assisted households to collect data on changes in income and 

housing costs. 

To arrive at the affordability ratio, the PJ should divide the household housing cost by household gross annual income. For 

example: 

$700 monthly housing costs divided by $1,500 household monthly income = 47% of household income is for housing costs. 

Finally, the PJ can compare the percentage of income spent on housing in the new or rehabilitated home to the percentage of 

income spent on housing in the household’s prior situation. 

Special Considerations 

If applicable, include the cost of utilities from the household’s previous housing in any comparisons made in order to measure 

the benefits from more energy-efficient construction in making a participant’s new housing situation more affordable. 

If the agency develops different types of housing (for example, detached single family homes, multi-family attached 

townhouses), this indicator can show what type of housing is most affordable and cost-effective. It also may help identify 

which design features are the most efficient and desirable. 

As stated before, affordability must be examined in the context of local housing costs, since in many localities, the 

affordability standard of 30 percent of income does not reflect the reality of the housing market. Use of this indicator can 

spur discussion in the community on determining a meaningful measure of affordability. 

Lower housing costs can contribute to other improvements in quality of life by increasing cash available for other purposes, 

such as education, health care, transportation, and savings or investments. 

Finally, comparing the affordability ratio among families of different incomes and varying housing costs will generate the 

affordability range for housing in the community. 
Measuring Change in Property Values

Frequently, the intended outcome for housing programs that are designed to revitalize neighborhoods is to provide economic 

stability to the neighborhood. The value of a house, apartment building, or commercial property is determined by the price a 

buyer is willing to pay for it. When property values increase, the wealth or asset base of the community also increases. 

Why Use It? 

Strengthening the sense of community and improving safety and visual attractiveness will make a neighborhood a more 

attractive place to live, thus raising property values. An increase in property values that results from the activities of the

agency is strong evidence of success. 

Conversely, in communities experiencing skyrocketing property values or gentrification, documentation of the increases in 

property values may be needed to build the case for development of more affordable housing. 

How Do I Get the Information I Need? 

Since interpreting changing property values can be complex, it helps to evaluate whether different information sources show 

different trends. By considering different methods of data collection, one can gain a better understanding of the actual 

situation. 

Sources for information on property values include the census data (every 10 years), county records (generally every three 

years), and real estate agents. 

Once a year, a PJ can extract or calculate property values for the neighborhood from available sources. It is a good idea to 

track both median and mean (average) property values in the neighborhood if data sources allow both calculations. This 

permits control for extremes in the neighborhood so as not to distort data. 

Special Considerations 

To better understand the impact of external factors, PJs should compare property value changes in the target neighborhood 

to changes in property values for the city overall (or any other large area of which the neighborhood is part). 

It is also important to evaluate who is gaining the benefit from an increase in property values. If low-income residents’ 

homes are foreclosed or rents become unaffordable so that they have to leave the neighborhood, the target community does 

not benefit. 

When presenting data, PJs should discuss the causes of trends and note any inconsistencies between data from different 

sources. For example, if a major neighborhood employer or school in the community opens or closes, property values are 

likely to be impacted. Such events, and their effect on the data, should be reported and interpreted. 

When possible, it can be very useful to get separate medians and averages for different parts of the neighborhood. For 

example, if housing prices are going up in one part of the neighborhood while they are stable or declining in another part, 

people who know the neighborhood might be able to explain the reasons for the difference. 

Step 3. Select and Apply Measurement Tools 

A variety of tools can be used in the data tracking and analysis process. Spreadsheets are useful for tracking and 

managing program outcomes data that is numeric, particularly when the data is extracted from program records. 

Several new tools designed to capture information about people’s experiences and perceptions are also discussed in 

this section. These tools include observation, surveys, focus groups, and interviews. Naturally, some combination 

of these tools might be appropriate. Selecting the appropriate tool(s) depends on the agency’s measurement goals, 

the type of data it has decided to collect, and the amount of resources available to undertake data collection. 

Spreadsheets 

As discussed in Chapter 2, spreadsheets are an ideal tool for tracking and manipulating numeric data. Outcome data 

from primary and secondary sources can be recorded in spreadsheets and tracked overtime. Similarly, survey 

responses can be coded into numeric form and entered into a spreadsheet for easier manipulation and analysis. 

42
Observation 

Direct observation is an underused and valuable tool for collecting data. Observation provides the opportunity to 

document observable data without relying on the willingness of others to respond to questions. A significant benefit 

of observation is that it can be conducted during the course of routine activities such as inspections, monitoring 

visits, and community meetings. It is low in cost and can be done relatively quickly. 

Some examples of staff observations include windshield surveys of housing types and condition, the presence of 

vacant or abandoned units, and the number of “for sale” and “for rent” signs. It might also include information 

collected at community meetings, such as documenting how resident concerns change over time. 

To be useful and credible for performance measurement purposes, however, observations must be recorded as soon 

after the observation is made as possible. The use of standardized forms, checklists, field notes, and pictures for the 

collection and recording of observable data helps standardize this type of data. This is especially important when 

multiple staff members are involved in the procedure. It is always important to record the date and location of the 

observation and include a short description of the context in which the observation was made. 

In contrast, using observation as a tool to measure program outcomes has some disadvantages as well. Without 

ongoing coordination efforts, information can be biased by staff interpretation. It can also be difficult and time 

consuming to analyze qualitative data. 

Surveys 

A survey is a highly structured series of written questions that is administered to a large number of persons. 

Surveys provide information in a systematic and quantitative manner, which facilitates statistical analysis. One 

major advantage of surveys is that they collect otherwise unavailable data directly from users and potential users of 

programs and services. The only way to obtain information about a person’s needs, opinions, plans, expectations, 

and behaviors that are not directly observable is to ask the 

person about these issues. Collect what you need 

The number one rule of conducting surveys is “do 

not collect what cannot be used.” Short, wellfocused surveys are the most effective because they 

limit the time burden on respondents, which can 

help improve the response rate. For this reason, 

surveys should be designed only after it is 

determined what information cannot be obtained 

from other sources, and how the results will be 

analyzed and disseminated. 

For example, a PJ might want to evaluate the effectiveness of 

its rental rehabilitation development program. The PJ wishes to 

know if the neighborhood residents believe that the quality of 

neighborhood life has improved because of the development 

activity in the area. While there may be outcome indicators to 

help assess whether there have been positive changes in the 

neighborhood (such as the crime rate or the vacancy rate of 

residential and retail properties), the only way to determine 

how the residents feel is to ask them. A survey is an ideal tool 

for this purpose. 

While surveys are the most common tool for outcome 

measurement, they are also expensive and time-intensive. They are also practically useless if not done correctly. 

Agencies that do not have in-house data analysis expertise are advised to consult with or hire an outside analyst to 

help with both design of the survey instrument and the interpretation of results. Universities are an ideal resource 

from which to seek assistance. PJs who seek the assistance of outside experts should be sure to include them in the 

design of the survey instrument—not just the analysis of the results. It is also important to test the survey 

instrument prior to beginning data collection. No matter how well designed a survey instrument appears to be, it is 

only through testing the instrument that one uncovers problems such as ambiguous questions, inappropriate 

wording, and the need for questions to be added or eliminated. 

Finally, identifying and targeting the survey to the correct audience is critical and should be well planned. For some 

types of evaluations, it is equally important to talk to both participants and non-participants, such as contractors who 

do not participate in contracting opportunities, or families and households who either do not participate or drop out 

of programs. 

43
Focus groups 

Focus groups are in-depth guided discussions among several individuals led by a trained and unbiased moderator. 

They are a powerful means to evaluate services, identify needs, and test new ideas. Participants can be clients, 

prospective clients, partners, or members of the general public. 

Convening focus groups sends a powerful message to clients or prospective clients about agency commitment to 

excellence in service delivery. When it will not interfere with the participants’ comfort in speaking candidly, staff 

should be included to sit in on the meeting. Exposure to customers is especially useful for frontline staff. 

Whereas a survey can reach a large number of people to elicit a limited amount of specific information, focus 

groups reach only a handful of people (typically six to eight) to explore a topic in great detail. Ideally, participants 

should not know each other, as this provides an atmosphere in which participants are more comfortable to speak 

freely. Like surveys, however, focus groups need to be well planned, have a clearly defined purpose, and be 

targeted to a specific audience in order to generate useful data. 

The most important part of planning focus groups is to identify the purpose of the meeting and determine what 

information is being sought. In program outcome evaluations, a focus group might be helpful when seeking 

perceptions of residents (and other community members) about the impact of the program. Given the wide variation 

in the types of responses this line of questioning elicits, the in-depth nature of a focus group can generate 

information that is not attainable through other means. A focus group can also be used to help inform staff about 

issues so that a survey of a larger group can be more effectively designed. 

A focus group session typically lasts between one to one and a half hours, during which time no more than six 

questions are explored. Participants are informed in advance of the purpose of the discussion. Since these groups 

are small, it is important that participants show up. Offering a small stipend to cover any out-of-pocket expenses by 

participants, and/or making childcare available for those who need it, can improve the attendance. 

A discussion guide should be designed prior to the session. Questions must be designed and crafted carefully, and 

be focused on the purpose of the session. Questions must also be worded in a way that considers the needs and 

capabilities of the participants. For example, it may be necessary to use simple phrasing and avoid complex words. 

If non-native English speakers are participating, it is important to avoid catch phrases and slang. Questions must be 

worded in an objective way to ensure that the questions themselves do not “lead” a participant to provide a specific 

answer. 

Finally, effective focus group moderators steer the discussion to keep the group focused on the purpose of the 

meeting and ensure that no one or two people dominate the discussion. The facilitator should know enough about 

the work of the agency and the purpose of the evaluation to provide participants with enough background 

information to ensure they fully understand what is asked. Finally, participants should be provided a copy of any 

written summaries or reports that are generated as a result of the discussion. 

Interviews 

The final tool PJs may want to occasionally use is the interview. An interview is an in-depth conversation with an 

individual to collect specific information about his or her thoughts, perceptions, and experiences. With this tool, 

there is a small sample of interviewees who are typically not randomly selected. Therefore, it is not possible to 

extrapolate findings to people who were not interviewed. 

Interviews are helpful, however, in obtaining depth and clarification. One common use of this tool in service 

delivery programs is an exit interview with program participants to discuss what worked, what did not work, and 

how the program could be improved. Interviews can also help highlight individual versus group concerns, develop 

other research tools, and gather stories for marketing or public relations purposes. 

In-depth interviews generally begin with a written set of questions, although a good interviewer does not limit an 

interview to those questions or a limited set of possible answers. A good interviewer will, however, be sure to get 

answers to the initial and most critical questions. The interviewer must motivate participants to share their thoughts 

and feelings on the interview topic. 

44
Using interviews for collecting data to measure program outcomes is time consuming and expensive. In addition, 

analysis of qualitative data can be difficult. However, this method allows for depth in data collection—that is, the 

why’s and how’s behind the initial responses—and response rates are generally very good. 

Figure 3.1 summarizes the advantages and disadvantages of each data collection tool. 

45
Figure 3.4 

Advantages and Disadvantages of Data Collection Tools 

 Advantages Disadvantages 

Data Extraction from Program Records 

❑ Data collection is ongoing, allowing 

for analysis of productivity and 

performance 

❑ Relatively inexpensive (data 

collection can be incorporated into 

existing staff responsibilities) 

❑ You can amend procedures to 

collect needed information on 

future participants

❑ Value of data depends on how 

carefully it was recorded 

❑ Existing records seldom contain all 

data needed 

❑ Data from existing records may not 

be current 

❑ Generally do not provide postservice information on participants’ 

experience or conditions 

Observation 

❑ Can be conducted in the course of 

regular duties/activities 

❑ Low cost 

❑ Can be done quickly 

❑ Works well for observable 

indicators 

❑ Information can be biased by staff 

person's interpretation 

❑ Not appropriate for indicators 

where visual observation is not 

feasible 

❑ It is not always possible to have the 

same person making the 

observations 

❑ Analysis of qualitative data can be 

difficult and time consuming 

Surveys 

❑ Can collect otherwise unavailable 

data (e.g., data about a person’s 

opinions, needs, future plans, 

expectations, and unobservable 

behaviors) 

❑ Provides information that is current 

❑ Provides information in a 

systematic and quantitative manner 

(which facilitates statistical 

analysis) 

❑ Information can be biased by 

memory, interception, perceived 

pressure, and fears 

❑ If questions are not well designed, 

data may be useless (i.e., 

experience is necessary) 

❑ Response rate is generally lowest 

of all data collection methods 

Focus Groups 

❑ Relatively inexpensive way to 

explore topics such as how to 

improve service delivery, how to 

reach out to underserved 

populations, perceived needs, or 

customer satisfaction. 

❑ D o not generally allow for large 

data collection efforts (i.e., sample 

likely not representative) 

❑ Can deter discussion of sensitive 

topics 

❑ Do not allow for “outlier” concerns 

to be raised

Interviews 

❑ Allows for “depth” in data collection 

(i.e., can explore the “why’s” and 

“how’s” behind answers) 

❑ Response rate is generally good. 

❑ Data collection is time consuming 

and expensive 

❑ Analysis of qualitative data can be 

difficult. 

46
Step 4. Analyze the Data 

Program outcome data can be extremely difficult to analyze, particularly when using outcome indicators or using 

data that is derived from interviews, surveys, focus groups, or observation. 

Since the data collection process is so time consuming when using these tools, it is important to point out that data 

must be analyzed as soon after collection as possible. Data that gets “cold” is not reliable. Time for analysis must 

be built into the data collection schedule as soon as possible after the data has been collected. 

Analyzing survey data 

Typically, after a survey is completed, the raw data is entered into a spreadsheet or database (data entry) to assist in 

data manipulation and analysis. The data is then “cleaned.” In other words, the results are checked to ensure they 

make sense and are reliable. Data irregularities, which typically result from respondent or data entry errors, must be 

addressed to prevent skewed findings. This is especially important when the survey is self-administered by youth or 

persons with limited literacy skills. 

Statistical software packages are extremely helpful in analyzing survey data, but they can be complicated for new 

users. Deciding how to “cut” the data, searching for correlations, and identifying other relationships can be very 

complex. PJs should seek assistance from individuals with data analysis experience. 

Analyzing data from focus groups 

Focus group data is slightly easier to interpret because it is not quantitative, and there is ample opportunity for 

participants to explain their answers. Audio or video taping (with participants’ permission) can facilitate the 

analysis and augment session notes. Notes from focus group sessions should include an extremely detailed 

discussion of the information that was elicited, the nature of the participation in the group, and the perceived 

comfort level of the participants. One of the more useful pieces of information from a focus group is whether there 

were surprises in participants’ responses. When results are used in a formal report, it is prudent to share findings 

with participants before publication to ensure accuracy. 

Analyzing results from interviews 

The results of interviews, like focus groups, are somewhat easier to analyze because interviewees have expounded 

on the information they have provided. Upon completion of the interview, the results should be transcribed and 

discussed by a team of staff or other program partners. While some of the specific questions to be addressed by the 

group will be dictated by the purpose of the interview, in general, the discussion should focus on the following 

components: 

• Key findings—what did the interviewees say? Were there any surprises? 

• Were the interviewees in agreement? 

• Were there issues that the interviewees did not agree on? If so, are there patterns or explanations to those 

issues? 

• What questions are answered by the findings? 

• What questions are raised by the findings that might need to be explored at a future time? 

• What should the agency do with the findings (such as, change programs, conduct additional evaluations, 

etc.)? 

Some additional points in analyzing outcomes data 

When intended program outcomes are not achieved, it is sometimes difficult to determine why. As with 

productivity measures, issues associated with achieving effective outcomes can generally be grouped into one of 

three categories: (1) program design; (2) personnel/staffing; and (3) program processes and procedures. Examining 

the findings from each of these perspectives helps in thoroughly analyzing the situation and identifying all of the 

factors that might be at play. 

47
Often, there are many factors that impact program effectiveness that are not in the housing agency’s direct control. 

Achieving positive change for individuals, and improving distressed communities is challenging work, but 

investments of affordable housing dollars can make a difference. It is critical that those investments are made as 

effectively as possible within the limitations presented. Continually monitoring program effectiveness in terms of 

the agency’s ultimate goals, and not only in terms of the number of housing units produced, will improve housing 

agency performance. 

48
Chapter 4: 

 Bringing It All Together 

In prior chapters, productivity and program outcome measurement have been examined as two separate 

and distinct components of an overall system of performance measurement. Chapter 4 explores how these 

two components can be used together and summarizes the key lessons of the prior chapters.

Performance Measurement for Continuous Improvement 

The performance measurement process, when used for problem-solving and continuous improvement, is rarely as 

organized and straightforward as the previous chapters might suggest. Nonetheless, it is important for PJs to try and 

organize a system of measurement to keep the agency focused on both efficient and effective operations. Using the 

program outcome model, this effort can be realized. 

Performance measurement has been described as an important tool for diagnosing problems that are not otherwise 

easily understood. Chapter 2 describes how to identify productivity problems, collect data to measure production, 

analyze the data, and implement improvements. The goal of measuring productivity is to improve an organization’s 

efficiency in terms of the quantity and quality of “outputs” it produces. Chapter 3 describes the importance of 

measuring program outcomes to ensure that the intended program benefits are actually realized. 

Yet, performance-based measurement is not a one-time event; nor is it intended to be used exclusively to solve a 

particular problem at a particular time. Once a performance measurement system is in place, its purpose is to 

monitor progress on an ongoing basis so that changes and adaptations can be made to program operations to prevent 

future problems. 

Understanding the Relationship Between Program Outcome 

and Productivity 

It is not always easy to untangle productivity problems from issues impacting program outcomes. Consider the 

following relationships between program outcomes and productivity: 

• A program can achieve its intended outcomes but be grossly inefficient. Program outcome measures 

might indicate that a program is effective and is meeting the intended needs of the community. However, 

these measures will not reveal whether these outcomes are too costly, too time-consuming to complete, or 

inefficient. 

• Outcome measurement might reveal that a program is not effectively meeting the needs of the 

community, but without additional measurement, it may be impossible to determine why. Program 

outcome measures must be paired with productivity measures in order to understand where a program is 

falling short and why it is not achieving intended goals. This is particularly true when the intended outcome 

will be impacted by the quantity of the product generated. For example, consider a homeowner-occupied 

housing rehabilitation program for which the intended program outcome is to revitalize a particular 

neighborhood. The PJ measures a variety of outcome indicators on an annual basis and determines that the 

program is not having an impact on the neighborhood. This lack of impact may be the result of the program 

not generating a sufficient number of rehabilitation jobs to impact the community. Alternately, the lack of 

impact may be the result of a program design that is inadequate for the target neighborhood because there are 

many other types of blighted property (e.g., large rental properties or commercial properties) that remain 

untreated. Further analysis will be required to make the determination of cause. 

• A program may be meeting its production objectives, but may not be achieving the desired program 

impact. It is certainly possible that an efficient, well-administered program may not be effective in meeting a 

PJ’s desired outcome, nor be the most suitable program for meeting a PJ’s goals. 

PJs need not abandon programs whose outcomes are not achieved, or are achieved in only a limited way. However, 

PJs should remain open to creating new partnerships, making program modifications, or undertaking additional 

49
initiatives to more effectively reach their goals. For multi-faceted problems, PJs should forge partnerships with staff 

of other local and state government agencies (such as economic development agencies, law enforcement, and social 

services providers) to determine what other issues might be preventing the program from meeting its intended goals. 

While PJs may not have complete control over all variables impacting the neighborhood, it may be able to improve 

its impact by coordinating efforts with others that share similar goals. 

The in-depth case studies that follow demonstrate three different ways PJs can use performance measurement. The 

first case study illustrates how a state initiates a performance measurement system to measure the performance of 

its recipients and subrecipients. The PJ’s first attempt at using performance measurement is focused on determining 

whether the state’s housing partners are meeting production goals. The state will use the data it collects to 

strengthen program decisions and evaluate its partners. 

The second case study describes a city in crisis—the city’s CHDO partners are not developing the rental units they 

have been approved to develop in a timely manner, and the city is at risk of losing its funding. In this case, the city 

uses performance measurement methods to identify information that it needs to be collected on an ongoing basis to 

avoid similar problems in the future. 

The last case study illustrates how performance measurement can be used to keep a PJ’s program on track. In this 

case, the PJ has been using performance measurement to evaluate its progress in revitalizing three city 

neighborhoods. The performance data suggest that the city is not achieving its desired impact. The PJ uses this 

information as the basis for additional evaluation and problem-solving that leads to improvements in its operations. 

Together, these case studies illustrate the diversity and flexibility of performance measurement and the benefits that 

can be achieved when measurement activities are incorporated into program operations. 

Case Study 1 

Initiating a Performance Measurement System at the State Level

This case study demonstrates how one state agency implements a performance measurement system in its special 

needs housing program. The state requires that recipients and subrecipients use the state’s program objectives and 

milestones as the basis of regular reporting so that it can track the progress of its partners. As this basic 

performance is measured, the state decides to collect additional data to help it assess its efficiency and effectiveness 

in program delivery. 

Background 

Marshall Goodman, Executive Director of the State Department of Housing, decides to implement performance 

measurement as a tool for managing the state’s limited housing resources. He chooses to focus on state recipients 

and subrecipients who participate in the state’s special needs housing programs before instituting these changes 

agency-wide. He targets the special needs housing division because this division has been very successful at 

building coalitions and collaborating with its housing partners, and it has done an exemplary job with strategic 

planning. Mr. Goodman believes these attributes are good indicators that this division might be successful in 

implementing performance measurement. 

Building Staff Support 

Mr. Goodman convenes a working group of staff in the special needs housing division and its partners to plan for 

these changes. To initiate the performance measurement program, Mr. Goodman asks the team to identify the 

program’s quantifiable objectives, and then to outline the milestones that must be accomplished in order for each 

objective to be met. 

He knows that the state’s new system of measuring performance will require a different mindset among his own 

staff and community development partners. To begin the process, he disseminates information that describes how 

other states have used and benefited from performance measurement. He also shares information on how using 

performance measurement might affect local housing programs. In his educational efforts, Mr. Goodman stresses 

50
that the ultimate goal of performance measurement is to improve the efficiency and effectiveness of local programs 

for the benefit of the client—not to punish recipients. To allay any concerns stakeholders may have, he promises 

that in the first three years, results will not be used to penalize any partner, but rather to better target technical 

assistance and improve capacity-building efforts. Finally, Mr. Goodman assures the state’s housing partners that the 

state will establish a feedback system so the partners can provide feedback on how the system is working. The state 

is committed to being flexible and open to modifications as the program is carried out. 

The staff voices many concerns about the difficulties involved in measuring performance in an area as complex as 

special needs housing. They balk at establishing timelines for the objectives, arguing that barriers related to 

NIMBYism, layering multiple sources of financing, and coordinating with social service providers can all impact 

timeliness. Many staff members express fear that if recipients are not able to meet the state’s objectives in a timely 

manner, it might be interpreted as a reflection on their ability to monitor and manage the projects. 

In spite of the resistance, Mr. Goodman is steadfast in pursuit of developing a more standard process for measuring 

the progress of the agency, and in particular, its recipients and subrecipients. Nonetheless, he listens to and 

acknowledges staff concerns and reassures them that the state will continue to collaborate with its partners in order 

to improve performance. He is particularly aware of the difficulties involved in measuring certain types of 

qualitative information, so he solicits staff input to identify ways to capture the qualitative information that might 

affect performance. 

Identifying Objectives and Milestones for Measurement 

The state’s special needs housing program has a relatively simple design. The state uses its HOME funds to 

develop housing that will be used to serve persons with special needs. The state makes funds available through a 

Request for Proposals that is issued twice per year. Recipients can use funds for acquisition or development of a 

variety of types of housing, including single-family homes, apartments for independent living, single room 

occupancy (SRO) housing, or group homes. To be eligible for HOME funds, applicants must have a viable 

business plan that demonstrates how they will cover operating costs for the units (since most residents will be able 

to pay little or no rent). Applicants must also demonstrate that they have a supportive services plan in place that is 

appropriate to the target population. 

The performance measurement team decides that, for expediency, it will establish initial program objectives by 

evaluating the state’s Consolidated Plan as well as the gaps analysis from the state Continuum of Care application. 

Based on its analysis, the team establishes the following state objectives: 

• Create 200 units of permanent supportive housing for persons with special needs; 

• Maintain ongoing housing stability by providing/coordinating case management and supportive services for 

all clients that require them. 

In order for the state to meet its objectives, there are a number of milestones that the recipients must meet. The 

state decides that it must establish some general milestones for data collection and analysis purposes, even though 

not all local program designs will fit within the milestones perfectly. It outlines the following milestones: 

• Objective 1: Create 200 units of permanent supportive housing for persons with special needs.

— Milestone #1: Conduct site-related pre-development 

activities. 

Defining Milestones 

Recognizing that each local program is 

designed differently, the state staff decides to 

identify and track a core set of milestones that 

is common to all programs (e.g., predevelopment, construction, occupancy). It 

identifies the activities under each milestone 

that must be complete before the stage is 

considered complete. Although different 

program designs might allow for some steps to 

be carried out concurrently, this type of 

reporting system allows for some flexibility in 

how each recipient defines each stage. By 

aggregating the level at which its partners are to 

report, the state also minimizes the reporting

 - Conduct environmental review. 

 - Identify site(s) and secure site control. 

 - Approve architectural plans, if needed. 

— Milestone #2: Conduct project-related pre-development 

activities. 

 - Secure social services funding. 

 - Secure development funding. 

 - Secure operating funds. 

51
What will performance data tell the state? 

The state expects that the data it collects about the 

status of its projects in the pipeline will: 

• Help it understand the housing needs of its special 

needs residents and the development needs of its 

special needs housing providers. This information 

will be used to improve planning and program 

implementation in the future; 

• Compare the performance of its various providers; 

and 

• Highlight areas where providers are having 

difficulties, and thereby identify issues that might 

require technical assistance. 

 - Finalize and approve underwriting. 

 - Conduct program compliance review. 

 - Conduct loan closing. 

 - Select property management firm, if needed. 

 - Select supportive services provider, if needed. 

— Milestone #3: Complete construction. 

 - Issue order to proceed and start construction. 

 - Develop and adopt operating policies and 

procedures. 

 - Complete 50 percent of the construction. 

 - Complete 100 percent of the construction. 

— Milestone #4: Establish occupancy. 

- Identify and recruit applicants from existing waiting list(s) and through referrals from human services 

organizations, shelters, the criminal justice system, hospitals, and street outreach programs. 

 - Review applicant file or letter of referral and determine applicant status. 

 - Screen and interview applicant. 

 - Determine eligibility of applicant. 

 - Eligible special needs occupant moves into unit(s). 

• Objective 2: Maintain ongoing housing stability by providing and/or coordinating case management and 

supportive services for all clients that require them. 

— Milestone #1: Conduct client needs assessment and develop supportive services plan. 

— Milestone #2: Secure necessary services (through direct provision or by referral). 

— Milestone #3: On quarterly basis, confirm continued occupancy, evaluate delivery of services, reassess 

client needs, and update supportive services plan. 

Involving Partners 

With the performance objectives in hand, the Executive Director and state staff meet with the state’s housing 

partners (that is, its recipients and subrecipients). At these meetings, the state staff explains the importance of 

performance measurement and describes the process the state is using to implement the change. The state solicits 

feedback from its partners, listens carefully to their concerns, and tries to respond openly to allay any fears that 

arise. The state’s partners express fear that there is considerable time and effort involved in reaching each of the 

milestones, and that measuring only the achievement of the milestone is not reflective of the amount of effort 

exerted. The state encourages its partners to create “sub-milestones” for their own internal use as a way to track 

progress toward the state-established milestones. The state staff makes clear, however, that the state’s objective is 

not to impose additional administrative burden on them, but rather to develop units and serve clients more 

effectively. 

Once several group meetings have been held, state staff members convene meetings with each recipient or 

subrecipient individually, to discuss the specific expectations of each housing partner. In these one-on-one 

meetings, the PJ and the housing partner discuss the partner’s program design and negotiate the specific activities 

that will be undertaken by the housing partner that are included under each milestone. The meetings are also used 

to establish quantitative goals to be met by the partner, timelines for completing each milestone, reporting 

requirements, and language for the written agreement or contract between the two parties. 

Following the series of meetings, the state staff finalizes a standard reporting form for recipients and subrecipients 

to facilitate data collection on each of the identified milestones and objectives. It anticipates requesting quarterly 

reports from its recipients and subrecipients to assess progress towards the objectives. Information Systems staff is 

engaged to expand the division’s existing database to capture this new information. 

52
Implementing the Process 

The state’s final reporting form captures data from each recipient and subrecipient about when each milestone is 

reached. Measuring progress regarding the development objective is relatively straightforward, and the state is able 

to aggregate the data for all reporting recipients and subrecipients. It plans to generate the following information 

each year: 

• The number and percentage of all developments that have reached each milestone. This core 

information will indicate the state’s progress toward its overall goal of creating 200 units of housing. 

• The average time between each of the following stages:

— Project selection; 

— Loan closing; 

— C onstruction start; 

— Construction completion; and 

— M ove-in. 

 The timeliness data is important because it helps the recipient, subrecipient, and the state identify possible 

problems in the project pipeline. With this information, the state can assess if a particular partner might need 

additional technical assistance or support. Chronic difficulties by many housing partners might suggest 

systemic problems that the state can address on a wide-scale basis. 

• Actual annual operating expenses versus amount leveraged for operating expenses. Tracking annual 

operating expenses versus leveraged funds helps the state identify long-term sustainability problems early on. 

Over the long term, it will also help the state identify which types of projects are the most sustainable, and 

which recipients and subrecipients are most successful at leveraging funds. 

The data that is needed to assess the state’s progress towards its goal of providing social services to its residents is 

more qualitative in nature and, therefore, more difficult to standardize and capture. Nonetheless, the state will 

collect information in the following areas: 

• Number and percentage of applicants referred, approved, and rejected, and the reasons for rejections. 

This will reveal the extent of the state’s progress toward its goal, and will identify whether any particular 

special needs groups are left unserved. 

• The type of special needs addressed, tallied for each disability code, as a percentage of the total. This 

information will allow the state to continue its ongoing assessment of the needs in the state, including the 

populations who are served and the remaining gaps in the system. 

• The type and intensity of social services being provided to clients, tallied by service type and the 

average number of hours per client. This information, particularly when evaluated in relation to data on 

the average length of residency and reasons for leaving will help the state assess what types of housing are 

most cost effective, as well as what level of service(s) may be needed to help clients attain self-sufficiency. 

• Average length of residency and reason(s) for leaving. This data will help the state determine whether it is 

meeting its goal of providing long-term, viable housing solutions to persons with special needs. Tracking 

reasons for leaving the program is particularly important as it helps the state understand what is happening to 

participants when they exit the program (such as, are clients cycling back into homelessness or are they 

become self-sufficient?). 

To facilitate data collection, the state’s reporting form provides a list of common reasons for rejection, disability 

types, types of services, sources of income, and reasons for leaving. Each list is coded to facilitate the data 

collection process. In addition, the state has clearly defined each term, to ensure that all recipients are using the 

same definitions. The state is careful to coordinate with Homeless Management Information System (HMIS) efforts 

in the state to prevent any duplication of effort and to ensure terms are defined consistently across programs. 

53
Benchmarks 

In the first year, the state plans to focus on working with its partners to gain acceptance of the new system and 

measure progress toward the goals. Mr. Goodman expects that after the first year, the state will have a fairly 

accurate baseline of its performance and the performance of its partners. He plans to seek additional data from 

neighboring jurisdictions and national housing advocacy groups to learn about typical performance in the special 

needs housing field in order to compare the state’s performance to others in the field. This information will help the 

state formulate benchmarks against which it can evaluate its progress in coming years. 

Case Study 2 

Applying Performance Measurement to City’s Rental Housing Program

In this case study, performance measurement is used to identify and solve a rental housing production crisis. The 

performance measurement team is able to identify improvements that can be made immediately to address the 

crisis. It also develops strategies for data collection and analysis that can be used in the future to identify potential 

problems before they occur.

Background 

Grayson City is a large urban city. Under its rental housing production program, $1 million dollars is set aside each 

year for use by Community Housing Development Organizations (CHDOs). The program caps its subsidy at 

$10,000 per unit; development costs in the program average $40,000 per unit. Several organizations have been 

certified as CHDOs, but only one, the Bellefield CHDO, has actually been able to produce units. Other CHDOs 

have been approved for proposed projects, but the projects have not moved forward for a variety of reasons. The 

City is facing the possible recapture of CHDO set-aside funds in the coming year due to a failure to formally reserve 

and commit funds within two years and expend the funds within five years. 

Analyzing the Problem 

Based on information provided from his staff, Grayson City Housing Director Tom Osbourne has a general sense 

that only one local CHDO is having success producing rental housing units. Although he has not been very diligent 

about reviewing IDIS reports on a regular basis, he knows they provide a high level understanding of what is going 

on within the jurisdiction. As he suspects, a quick review of the IDIS reports PR27 and PR22 reveals that only one 

CHDO has produced units. His analysis of the reports confirms that the City is at risk of losing its CHDO funds. 

Tom had observed that his staff spends considerable time meeting with and conducting technical assistance visits to 

local CHDOs. The CHDO production levels were frustrating given the investment of staff time in assisting them. 

Tom writes two problem statements to get him started on defining the problem: 

• The City’s CHDOs, except for Bellefield, are not moving forward with projects in a timely way to use funds 

within HUD deadlines. 

• The City risks losing funds. 

It is clear by the nature of the problem that something is going wrong between the time the CHDO applies for funds 

and prior to construction start. To help him think about issues more systematically, Tom convenes a meeting of 

program staff to identify the key program milestones of that stage of the production process. After much discussion, 

the team develops the following list: 

1. So licit applications 

2. R eview applications 

3. Select projects to fund 

4. Not ify developer 

54
5. Work with developer to meet “construction start” criteria: 

— Site control 

— Development team identified 

— Environmental review 

— Market study 

— Non-discrimination and affirmative marketing plan 

— Financial underwriting 

— Program compliance review 

— Approval of preliminary plans 

6. Solicit approval of Loan Committee 

7. Go to loan closing 

In the process of articulating the milestones, Tom finds that staff is somewhat confused by the City’s own process 

and requirements. He also learns that some steps in the review process are conducted sequentially rather than 

simultaneously. In addition, Tom becomes aware of some tension between project management staff and other staff 

who are responsible for reviewing certain aspects of the project (such as the environmental review and preliminary 

plans). 

Identifying Possible Causes of Problems 

After outlining the production process, Tom asks his staff to help him brainstorm a list of possible causes. They 

organize their list into the following three categories: 

Effective Brainstorming 

Remember, when brainstorming possible causes of 

problems, all ideas should be included—even those that 

seem somewhat off-base or repetitive. When team 

members are not censored, they are more likely to open 

up and share their ideas and opinions. After all, accurate 

diagnosis of the problem is essential to effective problem 

solving! 

Program Design 

• CHDOs need working capital to get through the pre-development 

stage. 

• The City’s subsidy is not sufficient. 

• CHDOs need operating funds. 

• CHDOs need more or different technical assistance. 

• There are insufficient incentives or penalties provided for 

performance (or lack thereof). 

• The City has not done an analysis of need for affordable housing in the neighborhoods served by the CHDOs. 

• Program standards for projects need to be revised, and additional guidance about acceptable projects must be 

provided. 

• CHDO capacity for rental production must be developed. At the same time, the City must identify projects 

that are ready-to-go to prevent loss of funds. 

Personnel 

• CHDO personnel are not sufficiently experienced in housing development. 

• City program staff is not sufficiently trained to provide technical assistance to CHDOs. 

• City staff does not fully grasp the CHDO’s stumbling blocks and, therefore, are not able to using training 

time effectively. 

• CHDO staff is not motivated and are not capable of carrying out development activities. 

Process 

• Review process for selection of CHDO projects and/or CHDO partners is not adequate. 

• City requirements for funding are too stringent. 

• City staff does not attend CHDO Board meetings and are not aware of the CHDO’s obstacles. 

55
• The City review process is cumbersome, and deals die while waiting for City commitment. 

• The City’s application requirements and process are poorly defined, and CHDOs submit insufficient 

documentation with funding applications. As a result, City and CHDO staff spend too much time trying to 

develop a complete application package. 

• The City does not establish deadlines for CHDOs, nor does it emphasize the importance of timeliness and 

outcomes. As a result, CHDOs are focused on, and getting bogged down in, process issues rather than 

production. 

The team discovers that many of the potential causes they identified are highly interrelated. After further 

discussion, the team identifies the following two issues as the most significant and most likely to impact CHDO 

progress: 

• Insufficient funds. Some CHDOs had complained of insufficient funds to work through the predevelopment 

phase. The staff thinks that these concerns might be valid—that is, lack of funds might be preventing some 

groups from moving forward. 

• Ill-defined application process and submission of incomplete applications. Tom recognizes that the City 

staff had difficulty articulating the application requirements and process when mapping out the workflow. 

He shares his observations with his staff, and they agree that the City could improve not only its procedures, 

but also the way it informs its CHDO partners about the procedures. 

Collecting Data and Further Analyzing the Problem 

City staff examine each of the potential problems more closely: 

Potential Cause #1: Insufficient Cash Flow/Funds 

Exploring the cash flow issue is a high priority for Tom because he believes that if the projects are insufficiently 

funded, the City can address the current crisis by providing additional funds to the projects. As a matter of policy, 

however, he does not want to create the impression that the City will “bail out” groups that are not able to leverage 

private assistance for their projects. Further, there is strong political pressure to leverage private funds for HOME 

projects—a policy he himself has encouraged. Therefore, if changes are needed, even temporarily, the changes 

need to be based on a sound rationale and accurate data. 

Given the City’s $10,000 per unit cap on assistance, a CHDO needs to leverage, on average, $30,000 per unit for its 

projects, for a leverage of 3:1. The City staff recalls that the cap had originally been instituted in the program to 

ensure that there was private backing of the City’s projects. However, the $10,000 limit was established somewhat 

arbitrarily. As a result, the City staff felt that it needed to assess whether the leverage requirement is reasonable. 

Further, the City wants to determine whether CHDOs have sufficient working capital to cover pre-development 

costs, such as site acquisition, environmental review, preliminary plans, and underwriting. To help with its 

assessment of these issues, City staff identifies the following information and sources of information to review: 

• A current statement of project commitments from all funding sources to assess the financial status of the 

approved projects (from CHDO operational and project records); 

• The identified funding sources for predevelopment activities (from the CHDOs’ project development 

budgets); 

• Recent monthly financial statements (from each CHDO); 

• Amount and type of non-HOME funding sources for the Bellefield CHDO project (from its project records); 

and 

• Average percentage of private funds leveraged for projects in neighboring jurisdictions (from a telephone 

survey of those jurisdictions). 

The City develops a form on which to capture the data it collects. For each of the CHDO projects, it wants to 

compare: 

• Total project funding sources to secured project funding sources; 

• Desired leverage ratio (3:1) to actual leverage ratio; 

56
• Anticipated predevelopment costs to total funds available for predevelopment activities (either project 

commitments or other unrestricted organizational funds). 

The City runs into many problems when it tries to conduct its analysis. First, the CHDOs do not use a standard 

development budget format, so it is extremely difficult for City staff to compare costs and projected funding sources 

with any accuracy. For some projects, the City does not even have a complete development budget! The City staff 

realizes that it has been making project commitments to CHDOs far too early in the process—well before the 

CHDOs are financially ready to work on the projects. Working through this issue brings to light the significance of 

the second issue the team has identified—an ill-defined application process and project selection criteria. 

The staff evaluates the information it collects to the extent possible. It finds that the projects, on average, have 

secured only 75 percent of the necessary development funds needed to carry out the development projects. This is 

consistent with its finding that neighboring jurisdictions require a more modest 2:1 leveraging commitment than the 

City of Grayson’s 3:1. Further, the staff finds that the City’s one successful CHDO, Bellefield, secured its 

development funding by capitalizing on an unusually good relationship with a small, private foundation. 

Staff also learns that most CHDOs have sufficient predevelopment financing. However, some of the newer 

organizations were struggling financially. 

As a result of the information it has collected and analyzed, the staff recommends several policy changes: 

1. Increase the cap on assistance from $10,000 to $20,000. This recommendation will enable most of the 

CHDOs to move forward immediately with their projects. 

2. Begin tracking the leverage ratio for projects that go to loan closing. This information will help the City 

monitor its success at leveraging private funds for its projects. 

3. Make HOME funds available to newer CHDOs in the form of predevelopment loans and operating funds to 

help them finance the predevelopment phase. 

4. Develop a standard and consistent project application package with standard forms and underwriting 

criteria for CHDOs. 

5. Develop and implement standard project selection criteria. 

6. Develop a performance-based project monitoring system for CHDO projects to prevent the crisis from 

recurring in the future. 

Potential Cause #2: Ill-defined Application Process and Submission of Incomplete Applications

After working through the first issue, it is clear that the City’s production problem could have been minimized or 

avoided if it had standard application forms and a standard application process, and if it conducted a more stringent 

review of CHDOs prior to selecting and approving projects. Tom asks his staff to undertake a structured review of 

the CHDOs’ applications from the last funding cycle. Specifically, staff looks at: 

Not everything needs to be measured! 

While working through its CHDO 

production problem, the City staff are 

identifying a combination of (1) 

information they can analyze in the short 

term to have a better sense of the 

problems and possible solutions; and (2) 

information they should track over time 

to monitor progress and avoid the 

problem in the future. 

At this stage, it will be tempting to want 

to track many things over time, but it is 

important to be judicious in deciding 

what really needs to be monitored, and 

what simply needs to be measured once 

and analyzed while solving a particular 

problem.

• The CHDO application package and required elements of funding; 

• The submitted proposals; and 

• The CHDO selection criteria the City used to select CHDO projects. 

Upon review, the staff finds that the City’s CHDO application process focused on 

determining CHDO eligibility instead of project feasibility. As a result, the 

submitted proposals provide only cursory information about the proposed project(s). 

Interestingly, the City has a far more stringent selection process for its private 

developers. Early on in the HOME Program, however, the City had so few 

qualified CHDOs it relaxed its standards to encourage participation. As CHDOs 

organized and developed capacity, the City had not modified its selection process 

accordingly. 

57
The staff also identified a number of specific problems: 

• Critical HOME program requirements that impact project costs, such as HOME rent structures, long-term 

affordability requirements, Davis-Bacon requirements, and accessibility requirements, are not articulated in 

the application package and, therefore, not properly accounted for in the proposed budgets. Without 

identifying these items, the proposed development budgets were inaccurate. 

• The City did not require CHDOs to provide evidence of development capacity nor rental management 

capacity in their applications for funding. Therefore, neither factor had ever been evaluated by the City. 

• The City stated no preference for, nor did it ask for information about, projects that had secured other 

funding. As a result, City funds were often the first to be committed to projects. 

It is clear that the City needs to re-define its application process. As a first step, the team compiles a list of required 

items for future applications: 

Managing Change 

When implementing change that is not universally 

popular, it is especially important to involve key 

stakeholders, including community and elected 

leadership. Using the data collected during the 

measurement process can help educate and persuade 

stakeholders of the need for improvements. In the 

City of Grayson, Tom Osbourne suspected that 

several CHDOs would have a difficult time meeting 

the new standards, and as a result, they might 

pressure the City to abandon its efforts. Tom 

Osbourne prepared special reports highlighting the 

loss of time, and potential loss of funds, that resulted 

from the lack of monitoring in the prior year and 

shared this information with key stakeholders. This 

information was critical in persuading and securing 

th t f th l t d l d hi f th Cit

• Evidence of site control; 

• Sources of other funds, level of commitment from other funders, and 

amounts committed or projected; 

• Capacity and experience of development team; 

• Capacity and experience of rental management entity; 

• Community support and/or involvement; 

• Development budget (in standard format), based on HOME rents and 

City cost control guidelines, and in compliance with per unit subsidy 

limits; 

• Fifteen-year operating pro forma (in standard format); 

• Plan to meet economic opportunity requirements; 

• Description of project significance and of market analysis; 

• Preliminary plans, if available; and 

• Project timetable. 

In addition to evaluating each CHDO’s capacity at the outset of a project, the City also recognizes the need for a 

system to monitor each CHDO’s progress throughout the development process, particularly during the predevelopment phase. Under its existing process, the City lacks a mechanism for identifying when and where in the 

process CHDOs encounter problems, and what type of technical assistance might be needed. 

With its workflow analysis, the staff re-defines the milestones for each phase of the production process. At this 

stage, staff also establishes a set of preliminary benchmarks for how long it thinks each stage of the process should 

take. It will use these estimates to measure CHDO progress in the first year. The City plans to measure the average 

completion time for each step in the process, and in the future, it will use those actual averages as benchmarks 

against which to measure CHDO performance. The City recognizes that countless factors can impact timeliness, 

such as seasonal variations, subcontractor availability, and availability of supplies. However, it also has learned the 

hard way that without a mutually-agreed upon schedule, no progress will be made. 

Prior to implementing these changes, the City convenes a number of meetings with its CHDOs to explain its plans, 

discuss the specific benchmarks it will use, and solicit feedback and concerns. Initially, these meetings are 

convened with open attendance for all CHDOs in order to get the broadest perspective on its performance 

measurement system. It refines its performance criteria as a result of this process, and incorporates the criteria in its 

next Request for Proposals. As part of the criteria, the City establishes a six-month deadline for all projects selected 

under the solicitation to be approved for loan closing. 

Once the City selects proposals, it meets with funded CHDOs individually to negotiate specific performance 

measures and deadlines based on the current status of each project and the CHDO’s own proposed timelines. The 

agreed-upon benchmarks provide staff and CHDOs with a clear understanding of expectations. Moving forward, 

the benchmarks provide an objective basis on which to monitor CHDO progress. 

58
Conclusion 

Most performance problems are the result of many, often interrelated causes and solving these problems and 

working to improve performance is an ongoing process. One of the most difficult steps is identifying the cause(s) 

of a problem when there is no or little information about it. By beginning with whatever reasonable data is 

available to solve the problem, a data collection strategy can be developed. As additional information is collected 

and analyzed, the hypotheses about the cause of the problem can be re-evaluated and revised. This process typically 

leads to the identification and resolution of the related problems, as it did in Grayson City. 

Case Study 3 

Applying Performance Measurement to City Neighborhood 

Revitalization Program

In this case study, the City of Honeyvale has been using performance measurement to track and measure the 

program impact, or outcomes, of the city’s neighborhood revitalization program. The outcome indicators being 

used by the City of Honeyvale indicate that the City is not meeting its overall program objectives. It uses this 

information to energize a team of community members and staff to analyze the problem, uncover why there has 

been little impact, and make changes to improve the program’s impact in the future. The City draws on a 

combination of the performance measurement tools and techniques discussed in Chapters 2 and 3.

Background 

The City of Honeyvale is spending its HOME funds in a timely manner, completing many projects across a range of 

homeownership and rental activities. The current goal of its program is the stabilization and revitalization of three 

target neighborhoods. Each year, as the City prepares its Consolidated Plan, it evaluates the impact of its 

neighborhood revitalization program by measuring three outcome indicators: 

1. Change in property values in terms of average market rents and average property sales prices, as measured 

by the City’s assessment division; 

2. Rate of abandoned properties, as measured by City staff windshield surveys; and 

3. Level of private development activity, as measured by the number of building permits issued for non-public 

development. 

The City of Honeyvale has been conducting performance measurement for the past four years. For the fourth 

consecutive year, it has found no significant change in any of the indicators. In addition, neither the staff nor key 

community leaders perceive the expenditures to be making an impact in terms of neighborhood stability or 

appearance in the City’s target neighborhoods. 

Staff and residents have expressed concern that City-funded rehabilitation jobs have deteriorated quickly, and that 

city-assisted new housing has not changed the nature or overall appearance of the neighborhoods in which it is 

located. In some cases, the new housing has also deteriorated quickly. 

Since the City of Honeyvale has been conducting performance measurement for four years, it knows the following 

facts: 

• It has invested $18 million in the last four years in three different neighborhoods. 

• Approximately 25 percent of its funds were invested in vacant property rehabilitation for new homebuyers, 

50 percent in a homeowner rehabilitation program, and the remaining 25 percent in small rental property 

rehabilitation. 

• There is no observable or measurable impact in the neighborhood. 

The Executive Director of the Honeyvale Housing Development Agency, Daisy Settles, recognizes that change can 

be slow. Nonetheless, by the fourth year of the program, she had expected to see some indication that the City’s 

59
efforts were having an impact on the community. She meets with the mayor to discuss the agency’s findings, and 

the mayor reconfirms her commitment to “do what it takes” to turn the target neighborhoods around. 

Discouraged, but not defeated, Daisy convenes several meetings with a team of staff and community representatives 

to discuss the agency’s findings and explore what to do about them. After considerable discussion, the team 

concludes that the lack of changes in the outcome indicators is discouraging. However, the team recognizes that 

these indicators are influenced by many factors over which they have little or no control. The team is very 

concerned, however, about two factors that the housing agency does have control over and it believes that if it 

addresses these issues it might improve their effectiveness in 

revitalizing the neighborhood: Interagency Collaboration 

The team is convinced that the property values, vacancy rates, 

and level of private investment are all impacted heavily by the 

perceived crime problems, high unemployment, and insufficient 

supportive services in the neighborhoods. As the team 

proceeds with improving its own program, Daisy Settles meets 

with the mayor to recommend an interagency team to 

coordinate City services and improve comprehensive 

revitalization efforts in these neighborhoods. 

1. There is the perception that City-funded jobs often result in 

quickly deteriorating repairs; and 

2. City investments, including new housing, do not seem to 

make an observable impact in the community. 

In response to these issues, City staff generates a list of possible 

reasons for the lack of sustained, visible impacts resulting from the 

City investments: 

Program Design 

• The City’s rehabilitation standard may not emphasize exterior repairs, which could minimize the impact of 

rehabilitation work on the appearance, or curb appeal, of assisted properties. 

• The property exteriors may require additional work to ensure the durability of other repairs. 

• A higher level of subsidy might be needed for projects to promote durability and visual impact. 

• The City’s current strategy for targeting rehabilitation activities may not be effective—the neighborhoods 

may be too dispersed, or there may be too many target neighborhoods. 

• The target neighborhoods may be too deteriorated for the City’s rehabilitation program to impact. 

• The areas’ homeowners may not invest in their own properties and may not maintain them well. 

• The housing market may have changed since the program was initially conceived, and there may be a need 

for a different “mix” of investments in terms of homeowner, vacant property, and rental property 

rehabilitation. 

Personnel 

• Contractors may be performing substandard work or using cheap materials, and inspectors may not be 

performing adequate and frequent enough inspections to monitor and identify problems. 

• Staff may not be generating adequate work write-ups and plans (upon which contractor work is based). 

• Housing counselors that support the program may not be educating new homeowners about ways to maintain 

improvements to their homes effectively. 

Process 

• Project selection may not be strategic (in other words, properties that would have the most visual impact may 

not be identified and selected). 

• Projects may be selected for rehabilitation without adequate review and underwriting to promote long-term 

viability. (Note: This might also be a personnel problem.) 

• The City has not implemented a system for final quality control inspections and related follow up. 

• The City’s community and economic development programs may not be coordinated and may not target the 

same neighborhoods. This lack of coordination might reduce impact. 

• The City’s code enforcement efforts may not be carried out in a way that supports the investments in the 

target neighborhoods. 

60
In generating the list of possible causes, the staff realizes that a number of the issues are inextricably inter-related. 

It decides to evaluate two priority questions: 

1. Whether the work to the exterior of the properties is having an improved visual impact on the 

neighborhood; and 

2. Whether the HOME-funded development work is durable, and therefore able to have a lasting impact in the 

neighborhood. 

Measuring Visual Impact 

City staff decides that the first thing to look at is the visual impact of its development jobs because it has the biggest 

impact on the appearance of the neighborhood. It plans to determine the extent of exterior repairs that were 

completed, as well as those that are still needed, in order to evaluate whether its development had a positive visual 

impact in the neighborhood. It wants to evaluate this data systematically to determine whether, moving forward, 

there are items that it should incorporate into its rehabilitation standard to ensure that rehabilitation jobs are done on 

a more consistent basis. It also wants to generate a new cost estimate for the completion of those repairs that are 

still needed so that it can improve the visual impact of the previously funded properties. 

Staff knows that other cities sometimes hire consultants to help them conduct a formal study of past and planned 

rehabilitation jobs to determine the nature, extent, and cost of exterior repairs. It decides, however, to take a less 

expensive approach. Since survey work is time-consuming and staff resources are limited, the team develops three 

categories for the properties, based on strategic significance: 

1. The most visible, thorough-fare blocks of its three target neighborhoods; 

2. Secondary streets; and 

3. Dead end roads and infrequently traveled streets. 

The staff establishes survey teams to assess the visual attractiveness of each property in the first two tiers of the 

neighborhoods; it determines it is not practical to include the third tier. A staff member develops a simple checklist 

to facilitate the survey work, in order to make sure survey teams are collecting consistent and comparable 

information, and to ease the subsequent data entry process. 

The survey team conducts a visual assessment of every property on the targeted blocks. For properties that the City 

had developed, it also asks owners to assess the property. The surveyor lists all recommended improvements, 

regardless of his or her own opinion about feasibility or cost implications of the suggestions. For instance, the list 

includes inexpensive items such as cleanup, lawn reseeding, and fencing or enclosures (for example, to hide storage 

and trash), as well as more expensive items such as exterior siding, sidewalk repair, tree removal, or extensive 

landscaping. The surveyor also takes note of all existing vacant properties on each block. 

Once the surveys are complete, the city recruits two local real estate agents and an appraiser who frequently works 

on City projects to team up with the inspection staff to analyze the results. After looking at the survey results, the 

team is able to make several recommendations: 

1. Modify the city’s rehabilitation standard to ensure that cost-effective exterior improvements are performed 

and addressed for all properties in the future. The modifications include specific improvements that are 

most likely to improve curb appeal and market value. The team further recommends differential treatments 

to properties, depending on property location. Implementing this recommendation will require an increase 

in the program subsidy to allow for this work. 

2. Change the program design so that the City targets properties on the main thoroughfare for development as 

high priority, particularly the vacant properties. The survey team notes some City investments on 

properties where substantial vacancies remain. With a high level of vacancies on the block, the City’s 

investment has nearly no visual impact. The team recommends that on secondary streets and infrequently 

traveled roads where more than 10 percent of the properties are vacant, the City should not fund any 

development unless several properties are treated, leaving no more than 10 percent of the properties vacant. 

3. Use City funds to undertake specific exterior improvements for previously funded properties where such 

investments will provide substantial visual impact. If City funds are not available, the team recommends 

61
that the City request a waiver from the HUD Field Office of the HOME Program requirement that HOME 

funds cannot be reinvested after one year from project completion. This would enable the City to use 

HOME funds. 

4. Use CDBG funds to undertake a list of inexpensive treatments for non-funded properties that might have 

significant impact on the curb appeal of the block. The survey results revealed that a number of simple 

items, such as general clean-up and lawn re-seeding, could significantly improve the appearance of many 

blocks. This use of CDBG funds will support its HOME investments. 

The City also decides that in its annual block surveys, it will begin tracking the number of vacant properties in the 

target neighborhoods. In addition, it will begin to conduct curb appeal inspections. It will maintain the resulting 

data in order to conduct an ongoing assessment of the improvements to visual impact of the neighborhoods. 

Measuring Durability and Lasting Impact 

The staff determines that it needs to have factual information about the lasting nature of its repairs. As a result, it 

decides to review agency files for all of the investments of the past three years to determine whether the repairs 

made in the program quickly deteriorated after the initial rehabilitation or development. 

As a first step, the staff has to determine which units that received City development funding (rehabilitation or new 

construction) needed additional repairs within three years of completion of the initial development, including the 

type of repairs needed. It wants to evaluate several factors that might have had an impact on the repair records of 

those properties in order to analyze possible causes. Staff decides to collect data on the repair record of its 

properties and compare the data to other information it already tracks by project, including: 

• Location of units, by block; 

• Tenure type; 

• For homeowner-occupied units, whether the owner had participated in the property maintenance course of 

the housing counseling program offered (but not required) by the PJ; 

• Developer and general contractor; and 

• Inspector who signed off on construction progress and final completion. 

Without this information, City staff knows it would be difficult to gauge whether the City’s work itself is 

deteriorating, or if this is a misperception because of other deterioration in the neighborhood. For the rental 

properties, program staff decides to use the agency’s monitoring division files for data on maintenance of rental 

properties. This division has responsibility for carrying out the site visits that are required for HOME rental 

properties throughout the period of affordability. At the site visits, the city staff reviews repair information 

maintained by the rental management office. For homeownership housing, the agency decides to survey 

beneficiaries to determine what work the owners either have performed since the initial rehabilitation or currently 

need to perform. 

Community members express concern that by only using its 

own agency files to survey the rental properties, the housing 

department might not have full information about the scope of 

the problem. They feel that some occupants might have made 

repairs without notifying the management, that management 

might not have made repairs that were requested, or that some 

tenants might not have complained in the first place. As a 

result, the City agrees to conduct a tenant survey to determine 

the extent to which other needed repairs are not noted in its 

site visits. 

A staff member develops a simple form to use for compiling 

repair information out of the agency files. In addition, he designs a short, easy-to-read survey for the beneficiaries 

to complete about the nature of repairs completed or needed in their homes. At the same time, the staff computer 

Involving Diverse Constituencies 

Involving a variety of stakeholders in the data 

collection and analysis process is always a 

good idea because each group of stakeholders 

brings a different perspective to the process. 

Sometimes the diverse interests will require a 

skilled meeting facilitator, but when all 

stakeholders are included in the process in a 

meaningful way, the process will yield 

stronger, more reliable results. 

62
programmer creates several new data fields for the development database so that the collected information could be 

entered into the system. 

When the survey results were returned, the staff are surprised to note that there are many reports of problems from 

the tenants that were not uncovered in the agency’s monitoring files. Many of these were problems that tenants had 

not reported. 

Once the survey results are tallied and agency files reviewed, the new data is entered into the computer. The 

director generates a number of reports that reveal: 

• Approximately 36 percent of all rental units that received development funding required major repairs within 

the first three years following funding. 

• All but five percent of those units had been developed by the same contractor. 

• Nearly all the problems were related to poorly installed windows or siding. 

• There did not appear to be a correlation between the inspectors and the problem units, as all staff inspectors 

had inspected some of the units in question. 

• Occupants of deteriorating units that were owner-occupied had typically not received homebuyer counseling. 

In contrast, few owners that had received homebuyer counseling had problems with deteriorating units. 

• There did not appear to be any pattern related to the location of the units. 

It appears that there is a problem with units deteriorating shortly after completion, and it appears that much of the 

problem can be attributed to the quality of work being performed by one contractor, and perhaps one subcontractor. 

Staff meets with the contractor and shares its findings. Armed with accurate information and reliable data, staff is 

in a good position to address the problem head-on. The contractor confirms that one subcontractor performs the 

window and siding work for him, and this subcontractor had performed the work on the deteriorating units. At the 

City’s insistence, the contractor agrees not to use this subcontractor for any additional City work. 

At the end of this process, staff is convinced that tracking this information over time would help the City monitor its 

progress to ensure that these types of problems did not recur. It makes several recommendations to the Director: 

• Rental managers should be directed to complete undone repairs that were reported by the tenants, using 

reserves; 

• Since there were many repairs needed that had simply gone unreported, tenants should receive information 

periodically reminding them that all repairs should be reported to rental managers; and 

• Information about the agency’s repair record should be added to the agency’s development database and 

analyzed annually. 

Lessons Learned 

At this point in the process, the staff has a much better understanding of how reliable data and information can lead 

to improvements in its process. Several specific problems were identified: 

• Inadequate work by a contractor; 

• Insufficient investments in exterior work due to a minimalist rehabilitation standard; and 

• Inadequate targeting of properties for development. 

Although staff might have suspected any one, or all, of these problems, taking the time to collect data and measure 

the extent of the problems provides them with the information they need to make changes to program design and 

operations and track improvement efforts. As these improvements take hold, staff can re-direct resources to other 

types of improvement efforts. 

63
Conclusion 

An effective performance measurement system is comprised of two intertwined components: productivity and 

program outcomes. Performance measurement is important because it provides a structure for tracking key program 

data that can help determine (1) whether a housing program is successful; and (2) if a housing program is operating 

efficiently. This information can help managers make changes to improve program operations, thereby stretching 

HOME dollars further. In addition, measurement efforts provide a foundation on which to communicate program 

accomplishments, build support for programs, and build a motivated and effective team. Performance measurement 

is a cyclical process, not a one-time event. Most PJs track certain kinds of information, but systematic data 

collection and analysis can help a PJ prevent problems in the future by monitoring changes in production or 

program outcomes. 

When there are any issues or concerns with production or effectiveness, PJs can undertake a process to identify the 

sources of that problem. This process includes the following steps: 

• Identifying and defining the problem; 

• Identifying needed data and data sources; 

• Measuring and tracking data; 

• Analyzing data and identifying areas for improvement; 

• Implementing improvements; and 

• Continuing measurement of progress. 

The measurement process is similar for both productivity and program outcome measures, although the types of 

data, data sources, and data measurement tools typically vary for the two types of analysis. Productivity 

measurement is typically built into program operations so that data is collected as it is generated. In contrast, 

program outcome measurement is more likely to be conducted on a periodic basis, often annually, to inform the 

strategic planning process. 

64
Appendix I: 

 National Housing and Community Development 

 Data Sources

1. Census Data 

The 2000 Census provides detailed information collected nationwide and can be manipulated to produce housing 

statistics. See the U.S. Census at www.census.gov. 

2. Decennial Data 

The U.S. Department of Housing and Urban Development (HUD) has requested that the U.S. Census Bureau 

tabulate Census 2000 data to create variables that are not available through standard Census products. These 

“special tabulation” data are needed to assist local government with housing planning as part of their Consolidated 

Planning process. The data may also be used in allocation formulas for distributing funds to local jurisdictions. See 

HUD User “Data Sets” at http://www.huduser.org/datasets/cp.html. 

3. Census 2000 Supplementary Survey (C2SS) 

The Census 2000 Supplementary Survey was a Decennial Census program designed to demonstrate the feasibility 

of collecting long form type information at the same time as, but separate from, the Decennial Census. It used the 

questionnaire and methods developed for the American Community Survey to collect demographic, social, 

economic, and housing data from a national sample of 700,000 households in 1,203 counties. Click on “Census 

2000 Supplementary Survey” in the title bar on the right-hand side of the U.S. Census, American Community 

Survey site at http://www.census.gov/acs/www/index.html. 

4. American Community Survey (ACS) 

The American Community Survey, when fully implemented, will survey 3 million households a year, located in 

every county, American Indian and Native Alaskan area, and Hawaiian Homeland, as well as in Puerto Rico. These 

data will provide detailed characteristics about our nation updated every year, rather than only once every ten years. 

The American Community Survey will provide estimates of demographic, housing, social, and economic 

characteristics every year for all states, as well as for all cities, counties, metropolitan areas, and population groups 

of 65,000 people or more. See U.S. Census, American Community Survey at 

http://www.census.gov/acs/www/index.html. 

5. American Housing Survey (AHS) 

The American Housing Survey Metropolitan Samples gives data on apartments, single family homes, manufactured 

homes, vacancy rates, family composition, income, housing and neighborhood quality, housing costs, equipment, 

fuels, size of housing unit, and recent movers. National data are collected every other year, from a fixed sample of 

about 50,000 homes, plus new construction each year. Select metropolitan areas are surveyed every 4-6 years, to 

measure local conditions. See HUD User “Data Sets” at http://www.huduser.org/datasets/ahs.html. 

6. A Picture of Subsidized Households 

This report and the accompanying data file sketch a picture of nearly five million subsidized housing units across 

the United States. It includes: (1) Totals; (2) Indian Housing; (3) Public Housing; (4) Section 8 Certificates and 

Vouchers; (5) Section 8 Moderate Rehabilitation; (6) Section 8 New and Substantial Rehabilitation; (7) Section 236; 

(8) Other HUD subsidies; and (9) Low Income Housing Tax Credits (LIHTC). Data are available for the 50 states, 

the District of Columbia, Guam, Puerto Rico, the Virgin Islands, and the Northern Mariana Islands. New versions of 

the report are released periodically. See HUD User “Data Sets” at http://www.huduser.org/datasets/assthsg.html. 

65
7. Fair Market Rents 

HUD releases Fair Market Rents (FMRs) annually. FMRs are used to determine payment standard amounts for the 

several housing programs. FMRs are gross rent estimates that include both shelter rent paid by the tenant to the 

landlord and the cost of tenant-paid utilities, except telephone. They are used to determine the amount of federal 

subsidy for recipients of housing aid. FMRs are set for all areas and provide estimated 40th and 50th percentile rent 

levels. See HUD User “Data Sets” at http://www.huduser.org/datasets/fmr.html. 

8. Income Limits 

Income limits determine the eligibility of applicants for HUD’s assisted housing programs. Income limits are 

calculated for metropolitan areas and non-metropolitan counties in the United States and its territories using the Fair 

Market Rent (FMR) area definitions used in the Section 8 program. They are based on HUD estimates of median 

family income, with adjustments for family size. Adjustments are also made for areas that have unusually high or 

low income to housing cost relationships. See HUD User “Data Sets” at http://www.huduser.org/datasets/il.html. 

9. U.S. Housing Market Conditions 

U.S. Housing Market Conditions, published quarterly, is a compilation of statistical data and written reports. Market 

conditions on the national level are presented for each quarter. Historical data are also presented in summary tables. 

Overviews of economic and housing market trends are presented for ten geographical regions, including a profile on 

a selected housing market. Each issue includes a summary of the overall trends in national housing and a topical 

piece that describes a particular, noteworthy aspect of housing activity. See HUD User “Periodicals” at 

http://www.huduser.org/periodicals/ushmc.html. 

10. State of the Cities Database 

The State of the Cities Database (SOCDS) provides data for individual Metropolitan Areas, Central Cities, and 

Suburbs. Detailed demographic, economic and housing data are available. Census data since 1970 is available, as 

well as a building permit database, employment statistics, and information on city and suburban government 

finances. See HUD User “State of the Cities Data Systems” at http://socds.huduser.org/index.html. 

11. Geographic Information System (GIS) 

The geographically coded data enables users to apply spatial analysis to a wide variety of housing and urban issues 

throughout the United States. There are geographic files with program data on American Housing Survey, 

Government Sponsored Enterprises and Home Mortgage Disclosure Act, Low-Income Housing Tax Credit, Picture 

of Subsidized Households, and State of the Cities. Additionally, there are boundary files for the Metropolitan 

Statistical Areas, central cities, and suburbs for the State of the Cities database. See HUD User “Data Sets” at 

http://www.huduser.org/datasets/gis.html. 

12. Multifamily Tenant Characteristics System (MTCS) 

Currently the public only has access to the Resident Characteristics Report of the MTCS. The Resident 

Characteristics Report summarizes general information about households who reside in Public Housing, Indian 

Housing, or who receive Section 8 assistance. The report provides aggregate demographic and income information 

that allows for an analysis of HA operations. See HUD’s Office of Public and Indian Housing’s “Resident’s 

Characteristics Report” at http://www.hud.gov/offices/pih/systems/pic/50058/rcr/. 

13. National Survey of America’s Families 

The National Survey of America’s Families provides a comprehensive look at the well-being of children and nonelderly adults, and reveals sometimes striking differences among the 13 states studied in depth. The survey provides 

quantitative measures of child, adult and family well-being in America, with an emphasis on persons in low-income 

families. See The Urban Institute, “Assessing the New Federalism, National Survey of American Families” at 

http://www.urban.org/Content/Research/NewFederalism/NSAF/Overview/NSAFOverview.htm. 

66
14. Home Mortgage Disclose Act (HMDA) Statistics 

The Home Mortgage Disclosure Act (HMDA), enacted by Congress in 1975 and implemented by the Federal 

Reserve Board’s Regulation C, requires lending institutions to report public loan data. The data include loan 

approval and denial rates by race, gender, income, and geographic location. See Federal Financial Institutions 

Examination Council “Home Mortgage Disclosure Act” at http://www.ffiec.gov/hmda/. 

15. National Association of Home Builders, Economic and Housing Data

The NAHB collects data on construction starts, homes sales and prices, state and local data, forecasts and economic 

impacts, multifamily housing index, and other resources. See National Association of Home Builders “Economic 

and Housing Data” at http://www.nahb.org/category.aspx?sectionID=113. 

16. National Association of Realtors Economic Research 

The NAR offers national data on existing home sales, economic trends, state data on housing markets, and 

metropolitan statistical area research on housing starts, vacancy rates, home prices, and other useful statistics. See 

National Association of Realtors “Research” at 

http://www.realtor.org/Research.nsf/pages/EconHousingData?OpenDocument. 

17. The Consolidated Plan 

The Web site contains archived consolidated plan executive summaries from jurisdictions in all 50 states and the 

District of Columbia. Consolidated plans are issued by state and large cities every five years, with updates issued 

annually. The plans include data on housing needs as well as other economic information. See HUD’s online 

library at http://www.hud.gov/library/bookshelf18/archivedsum.cfm. 

18. HUD Multifamily Insured Mortgages 

This database includes all HUD Multifamily insured mortgages except those from the Hospital Mortgage Insurance 

Program. The data includes the HUD project number, the project name and location, the number of units, 

information on endorsement, mortgage amount, payment, length and terms, mortgage holder, and mortgage servicer. 

The database is updated quarterly. See HUD’s Office of Housing, “Insured Multifamily Mortgages Database” at 

http://www.hud.gov/offices/hsg/comp/rpts/mfh/mf_f47. cfm. 

19. Low Income Housing Tax Credit Program 

The LIHTC program has been recently amended to give States the equivalent of nearly $5 billion in annual budget 

authority to issue tax credits for the acquisition, rehabilitation, or new construction of rental housing targeted to 

lower-income households. The database, created by HUD and now available to the public, contains information on 

nearly 19,700 projects and more than 935,000 housing units placed in service between 1987 and 2000. See HUD 

User “Data Sets” at http://www.huduser.org/datasets/lihtc.html. 

20. Section 8 Expiring Contracts Database 

This database provides a method to measure the potential impact of expiring project-based subsidy contracts in 

communities. It represents the most comprehensive picture of project-based subsidies. See HUD’s Office of 

Housing “Section 8 Contracts” at http://www.hud.gov/offices/hsg/mfh/mfhsec8.cfm. 

21. GovStats 

GovStats (formerly the Government Information Sharing Project) is an interactive website providing access to 

statistical data from the U.S. government. Its mission is to provide easy access to useful government data not 

available elsewhere on the Internet. It provides information on US Counties, Import/Export, the 1997 Census of 

Agriculture, the 1992 Economic Census, and other databases. See Oregon State University Libraries “GovStats” at 

http://govinfo.kerr.orst.edu/. 

67
22. Bureau of Labor Statistics 

The BLS is part of the U.S. Department of Labor and provides data on economic, labor and employment statistics. 

See U.S. Department of Labor, Bureau of Labor Statistics at www.bls.gov. 

23. State Data Centers 

Each state has Data Centers that are a source for census information and socioeconomic projections. This site is 

from the Maryland State Data Center. Data Centers from other states can be found by performing an internet 

search. See Maryland Department of Planning, Maryland State Data Center at http://www.op.state.md.us/MSDC/. 

24. Enterprise Foundation Resource Center 

This Enterprise Foundation site offers a database on model documents, how-to guides, program descriptions and 

templates for implementing community development programs. Also available is the Housing Developer Support 

System, Enterprise MoneyNet, Community Safety Virtual Campus, Community Forum and Job Board, Policy 

Information and Guides, software, publications and products, and several other resources. See Enterprise 

Foundation “Resources” at http://enterprisefoundation.org/resources/index.asp. 

25. National Neighborhood Indicators Partnership 

The National Neighborhood Indicators Partnership (NNIP) is a collaborative effort by the Urban Institute and local 

partners to further the development and use of neighborhood information systems in local policymaking and 

community building. NNIP partners have built advanced information systems with integrated and recurrently 

updated information on neighborhood conditions in their cities. The goal of NNIP is to democratize information, 

and participants concentrate on facilitating the direct practical use of data by city and community leaders. See The 

Urban Institute, National Neighborhood Indicators Partnership at http://www.urban.org/nnip/. 

68