DISCLAIMER

The contents of this report are the views of the contractor and do not necessarily reflect 

the views or policies of the U.S. Department of Housing and Urban Development or the 

U.S. Government.
Review of Energy 

Performance Contracts in 

Public Housing

Submitted by:

Anna E. Bower

Michael E. Canes

Stuart D. Funk

Jyothsna Prabhakaran

LMI

Amy Deora 

CIVIS Analytics

Rob Hazelton

Dominion Due Diligence Group
ii

Foreword

I am pleased to present this report on a national evaluation conducted by the U.S. 

Department of Housing and Urban Development’s (HUD’s) Office of Policy Development 

and Research. The study assesses the effectiveness and value of HUD’s Energy 

Performance Contracting Program administered by the Office of Public and Indian Housing 

(PIH). Energy Performance Contracts (EPCs) are a tool HUD makes available to Public 

Housing Authorities (PHAs). Over three decades, EPCs have enabled 250 PHAs, which 

include approximately 250,000 units, to secure private-sector financing to upgrade the 

energy efficiency of their housing stock by using the energy (and water) savings to pay 

for the improvements. 

The findings of this report have two main implications for more effective policy. First, the 

EPC program remains a valuable tool for PHAs that seek to maintain their properties 

within the public housing program. It is complemented by additional repositioning tools 

that HUD has made available through the Rental Assistance Demonstration (RAD) and 

more recently with reforms to Section 18 (Demolition and/or Disposition) and Section 22 

(Voluntary Conversion to Tenant-Based Assistance). Second, the EPC program 

continues to be less utilized by smaller PHAs, and further attention will be critical to 

enable these smaller PHAs to enhance their utility cost savings. HUD can support small 

PHAs by expanding the Moving to Work Program and improving access to utility 

partnerships, as described in the recently published PIH Notice 2018-20. Congress can 

also support small PHAs by passing the Economic Growth Act that will eventually allow 

small and rural PHAs to optionally freeze their utility baselines. 

The quantitative components of the evaluation found that PHAs having implemented 

EPCs reduced their electrical consumption more than twice as much as PHAs that have 

not. Researchers could not produce a similar analysis for other types of utilities, 

however, because of data-quality limitations in operating subsidy utility expense-level 

data. On an annual basis, HUD uses PHA-submitted data as well as Measurement & 

Verification (M&V) reports, which HUD requires to evaluate individual EPC performance 

and savings achievement to ensure proper provision of EPC incentives. These analyses 

provide substantial detail on individual EPC projects but do not allow for comparisons 

with other PHAs that did not pursue an EPC. Enhanced data collection policies and 

activities for utility expense-level data are needed to better measure the full benefits of 

EPCs when comparing EPC-using PHAs to non-EPC-using PHAs.

EPCs involve highly complex activities that have significant lead-time requirements and 

require partnerships with third parties. Safeguarding the public interest requires PIH’s 

Field Offices and Energy Team to exercise due diligence when analyzing a potential 

EPC project to ensure that the PHA will remain financially solvent after implementation. 

This in-depth analysis may deter participation by some Energy Service Companies 

(ESCOs) or PHAs. The study found that an important reason for PHAs declining to 

execute EPCs is the time and technical difficulty associated with the EPC process, 

despite significant training having been provided by HUD in the past. The EPC process 

is further burdensome for small PHAs that may lack the staff or expertise to fully manage 

the program. EPC participation across the public housing portfolio lags for small PHAs. 

Nearly 50 percent of large and medium PHAs have used EPCs, but only 7.9 percent of 

all PHAs and only 3.7 percent of small PHAs have participated. EPC is HUD’s only 
Foreword

iii

program specifically focused on energy, although tools such as RAD allow for the 

leveraging of significant resources that many PHAs are using to achieve energy savings. 

HUD is also aware that multiple other structural factors may constrain small PHAs from 

using EPCs. One such barrier is that ESCOs operate using a profit-driven business 

model, and the scale and complexity of EPCs for small PHAs may not provide them with 

sufficient profit potential or the ability to mitigate risk. Another factor is that third-party 

financiers may not find it advantageous to investigate and lend funds to small PHAs. 

Other options for pursuing energy savings outside of EPCs may be more suitable for 

small PHAs.

The study also shows that the adoption of private-sector financing and conversion of rental 

assistance models through RAD has increased exponentially in recent years. Although 

RAD (and now, the Section 18 and Section 22 alternatives) offers an ability to complete 

energy and utility efficiency improvements as part of broader project renovations, not all 

PHAs are interested in or are able to use these tools. HUD views EPCs as a mechanism 

that can continue to provide such PHAs with a means of tapping into private financing for 

sorely needed capital improvements. For this reason, the EPC program remains a valuable 

financing mechanism in HUD’s toolbox.

Overall, this evaluation illustrates that energy conservation efforts within HUD’s public 

housing stock will require policies that encourage and motivate PHAs to sustain 

partnerships with private capital financiers and the companies that deliver energy 

services—whether through EPCs, RAD, or other pathways. Through continued and new

efforts, more PHAs will be able to participate in these energy savings activities, resulting in 

better housing for the communities they serve.

Seth D. Appleton

Assistant Secretary for Policy Development and Research

U.S. Department of Housing and Urban Development
iv

Contents

Executive Summary ................................................................................................ES-1

Background and Method......................................................................................ES-1

Principal Findings ................................................................................................ES-1

Recommendations...............................................................................................ES-3

Chapter 1 Introduction...............................................................................................1-1

Chapter 2 Background...............................................................................................2-1

Chapter 3 Survey Sample ..........................................................................................3-1

Research Population ..............................................................................................3-1

Sample Design and Process...................................................................................3-1

Chapter 4 Survey Implementation.............................................................................4-1

Survey Response Rate...........................................................................................4-1

Corrections to the Data...........................................................................................4-3

Blank or Incorrect Responses ...........................................................................4-3

Discrepancies in Respondent Group Category Designations ............................4-3

Implications for Survey Response Analysis.............................................................4-5

Adjustment of Sample Sizes ...................................................................................4-5

Chapter 5 Survey Analysis ........................................................................................5-1

Statistical Weighting ...............................................................................................5-1

Statistical Hypotheses Testing—Comparing Group 1 with Group 3 ........................5-2

Development of Ratings and Averages.............................................................5-2

Calculation of Weighted Average Scores ..........................................................5-2

Hypotheses Testing ..........................................................................................5-4

Within Group 1 Differences—Units Subject to an EPC vs. Units Not Subject 

to an EPC .........................................................................................................5-5

Survey Evidence Concerning Why PHAs Did Not Undertake EPCs........................5-7

How to Improve HUD’s EPC Process .....................................................................5-8

Chapter 6 Telephone Follow-up Interviews ..............................................................6-1

Introduction.............................................................................................................6-1

Representation by Group and Size .........................................................................6-1

Survey Results .......................................................................................................6-1

Interviewee Recommendations...............................................................................6-3

Chapter 7 Analysis of Utility, Financial, and Physical Conditions Data .................7-1
Contents

v 

Background ............................................................................................................7-1

Analytic Approach...................................................................................................7-1

Analysis of UEL Data ........................................................................................7-1

Analysis of Long-Term Electricity Consumption Trends ....................................7-2

Analysis of Short-Term Electricity Consumption Trends....................................7-6

Analysis of Financial and Physical Condition ....................................................7-7

Chapter 8 Relationship of the RAD Program to EPCs .............................................8-1

Background ............................................................................................................8-1

PHA Options...........................................................................................................8-2

Gains in Energy Efficiency from EPCs and RAD.....................................................8-2

Implications.............................................................................................................8-3

Evidence.................................................................................................................8-4

Qualitative: Stakeholder Views..........................................................................8-4

Quantitative: EPC Approvals.............................................................................8-5

Chapter 9 Conclusions and Recommendations.......................................................9-1

Conclusions............................................................................................................9-1

Recommendations..................................................................................................9-2

Appendix A Responses to Survey Questions ......................................................... A-1

Appendix B Summary Notes from Survey Follow-up Telephone Interviews ........ B-1

Appendix C UEL Data Analysis ................................................................................ C-1

Appendix D Maps of Regions and of PHA Populations.......................................... D-1

Appendix E Abbreviations ........................................................................................ E-1

References................................................................................................................. R-1
vi

List of Exhibits

Exhibit 3-1. Mapping of HUD Regions to Census Regions......................................3-1

Exhibit 3-2. Group Population Counts.....................................................................3-3

Exhibit 3-3. EPC Sampling Strategy .......................................................................3-4

Exhibit 3-4. Non-EPC Sampling Strategy................................................................3-4

Exhibit 4-1. Full and Partial Response Rate............................................................4-2

Exhibit 4-2. Completed Survey Response Rate ......................................................4-2

Exhibit 4-3. Survey Responses by Region ..............................................................4-3

Exhibit 4-4. PHA Classifies Itself as Group 1 Whereas HUD Information

Classifies It as Group 2 or 3.................................................................4-4

Exhibit 4-5. HUD Data Classifies PHA as Group 1 Whereas PHA Classifies 

Itself as Group 2 or 3 ...........................................................................4-4

Exhibit 4-6. HUD and PHA Disagree on Whether the PHA is in Group 2 or 3 .........4-4

Exhibit 4-7. Final Numbers of Survey Responses by Group and PHA Size.............4-6

Exhibit 5-1. Survey Questions Relating to Utility, Financial, and Physical 

Condition .............................................................................................5-2

Exhibit 5-2. Ratings Attached to Answers to Utility-Related Questions....................5-2

Exhibit 5-3. Weighted Averages, Standard Deviations, and Standard Errors ..........5-3

Exhibit 5-3. Weighted Averages, Standard Deviations, and Standard Errors ..........5-4

Exhibit 5-4. Group 1 and Group 3 Differences in Average Utility Scores 

(* indicates statistical significance at the 95-percent level)...................5-4

Exhibit 5-5. Differences Between Average Financial and Physical Ratings 

(* indicates statistical significance at the 95-percent level)...................5-5

Exhibit 5-6. Percentage of Group 1 Respondents Who Would Do 

Another EPC........................................................................................5-5

Exhibit 5-7. Survey Responses on Utility Performance at EPC vs. Non-EPC 

Units ....................................................................................................5-6

Exhibit 5-8. Survey Responses on Financial Condition at EPC vs. Non-EPC 

Unitsa ...................................................................................................5-6

Exhibit 5-9. Survey Responses on Physical Condition at EPC vs. Non-EPC 

Unitsa ...................................................................................................5-6

Exhibit 5-10. Why PHAs Chose Not to Do an EPC .................................................5-7

Exhibit 5-11. Why PHAs Began but Did Not Complete the EPC Process................5-8

Exhibit 5-12. Ways to Improve HUD’s EPC Process Suggested by PHAs ..............5-8

Exhibit 6-1. Breakdown of Interview Sample by Group and PHA Size.....................6-1

Exhibit 7-1. Example Data Set and Calculated Averages for FL041........................7-2

Exhibit 7-2. Summary of Changes in Electricity Consumption Between EPC 

PHAs and Non-EPC PHAs...................................................................7-6
List of Exhibits

vii

Exhibit 7-3. Pre-, Post-, and Non-EPC PASS, MASS, and FASS Scores................7-9

Exhibit 7-4. Available Funds Over Time to EPC and Non-EPC PHAs ................... 7-10

Exhibit 8-1. Numbers of RAD Housing Units Authorized by Congress, 

by Year ................................................................................................8-1

Exhibit 8-2. Numbers of PHAs Executing EPCs, Converting to RAD or 

Doing Botha..........................................................................................8-2

Exhibit 8-3. EPC Approvals by Year .......................................................................8-5

Exhibit C-1. Data Modifications Made .................................................................... C-2

Exhibit C-2. Results of Statistical Analysis ............................................................. C-4

ES-1 

Executive Summary

Background and Method

The U.S. Department of Housing and Urban Development (HUD) seeks to assess the 

effectiveness and value of its Energy Performance Contract (EPC) program by 

comparing trends in the utility consumption and the financial and physical condition of 

public housing authorities (PHAs) that have implemented EPCs with those that have not. 

It also wants to discover the extent to which implementation factors affect the rate at 

which EPCs are used by PHAs. HUD contracted with a team headed by LMI (the LMI 

team) to obtain the comparative and implementation assessments.1

The study followed four lines of investigation.

1. Survey: An online survey was sent to a sample of several hundred PHAs. Using 

survey data, the team tested whether the PHAs that used EPCs outperformed 

those that did not with respect to reducing utilities and improving financial and 

physical condition.

2. Telephone interviews: The team interviewed 20 of the responding PHAs in more 

depth concerning their experiences with EPCs or improving their utility 

consumption without the use of this program.

3. Data analysis: Information from various HUD databases about PHA utility 

consumption and financial and physical condition was used to corroborate survey 

responses and statistically test the efficacy of EPCs in improving public housing 

utility consumption and financial and physical condition.

4. RAD impact: The interaction of HUD’s Rental Assistance Demonstration (RAD) 

program with the EPC program was assessed. The number of housing units in 

the RAD program and the number of EPC applications were compared to see to 

what extent the former may have replaced the latter.

Principal Findings

After analyzing the data obtained in the steps previously described, the LMI team 

reached the following conclusions:

EPC Savings

1. Statistical tests performed using HUD Utility Expense Level (UEL) data showed 

that PHAs that have implemented EPC projects have been able to cut their 

electricity consumption by more than twice as much as PHAs that have not

(10.3 percent as compared with 4.4 percent). This conclusion is backed at a 

95 percent level of statistical confidence.

2. Tests using UEL data for EPC users showed electricity consumption declined 

at a significantly greater rate in the years after an EPC was performed than 

in the years immediately before. This result held at a 95 percent level of 

1 The LMI team includes LMI, which headed the team, CIVIS Analytics, and Dominion Due Diligence Group. 
ES-2 

statistical confidence for all such PHAs as well as for small and large ones, but 

only at a 90 percent level of confidence for medium-sized PHAs.

Survey Results

3. Analysis of PHA survey data indicates those that performed EPCs reduced 

their use of natural gas and water significantly more than non-EPC users, 

at a 95 percent level of statistical confidence. Survey data indicated that EPCusing PHAs reduced their use of electricity and fuel oil by more than non-EPC 

PHAs as well, but those results were not statistically significant.

4. Survey data suggest that, during the past several years, those PHAs that have 

used EPCs have improved their financial condition somewhat more than 

those that have not. Surveyed EPC-using PHAs also indicated that the financial 

condition of units included within the EPC improved more than at their other 

units. These findings were not conclusive at a 95 percent level of statistical 

confidence, however. They also could not be confirmed through examination of 

administrative financial data.

5. PHAs that executed EPCs reported greater improvements in the physical 

conditions of their properties than PHAs that did not. The difference in 

reported improvements between the two samples was significant at a 95 percent 

level of statistical confidence.

6. Approximately 60 percent of PHAs that had executed at least one EPC 

indicated they were favorably disposed toward doing another. Because at 

least some of the other 40 percent that had executed at least one could already 

have accomplished all the energy investments they considered cost-effective, it 

is likely that more than 60 percent found the EPC program useful.

7. According to survey evidence, about two-thirds of PHAs that had executed an 

EPC at some units but not others indicated that the EPC resulted in lower 

utility consumption and better physical and financial condition than 

occurred in their non-EPC units. These responses furnish a strong rationale 

for why most of the EPC-using respondents expressed willingness to do another.

RAD Impact

8. The RAD program is being used by a large and growing number of PHAs. 

According to interview data, the RAD program offers more flexibility in the use of 

funds to address housing units’ physical needs and is easier to negotiate than an 

EPC. The annual number of EPC applications has declined over time as more 

PHAs have turned to RAD.

9. Few PHAs are likely to undertake both RAD and an EPC. Indeed, PHAs that 

have undertaken RAD often have bought out their EPC contracts.

EPC Challenges and Possible Responses

10. Of those PHAs that have never undertaken an EPC, the most important reason 

(39 percent) given was that such a project would not be cost-effective from their 

perspective. Other leading reasons were that RAD is a better alternative (20 

percent) or the process is too complicated (18 percent). 
Executive Summary

ES-3 

11. Of those PHAs that started but did not complete an EPC, the main reason given 

was that they did not have sufficient staff to see the project through. Another 

reason was that HUD’s EPC process is too cumbersome.

12. Interviewed PHAs indicated that HUD’s EPC program should be among the 

options available. Interviewees suggested, however, that because many PHAs, 

particularly smaller ones, do not fully understand the program, do not have the 

staff to undertake an EPC, or do not know how to deal effectively with 

contractors, more technical support would be useful.

13. Survey respondents and interviewees identified several means by which HUD 

could promote the use of EPCs. Such strategies include continued 

encouragement to undertake the program, direct subsidies, a streamlined 

application process, and greater technical assistance.

Recommendations

Based on these conclusions, the LMI team offers the following recommendations:

1. Because there is strong statistical evidence that EPCs have helped PHAs 

improve their units’ energy efficiency, the program appears to be accomplishing 

one of its principal purposes. If gains in energy efficiency remain an important 

HUD goal, then the EPC program should be maintained if not strengthened.

2. The program offers PHAs a means to tap into private capital markets. The RAD 

program also does so, but not all PHAs are interested in RAD or able to take 

advantage of it. For these PHAs, it is important to maintain the EPC program as 

an option.

3. According to some interviewees, HUD is no longer emphasizing its EPC program 

to PHAs to the extent it once did, and a number of PHAs are skeptical of 

undertaking it. These interviewees assert that increased communication 

regarding the program, particularly with respect to lessons learned and how to 

deal with contractors, would lead more PHAs to consider the program. In 

addition, several interviewees believe that more financial and technical support, 

particularly directed toward smaller PHAs that seek to self-manage EPCs, likely 

would increase participation.

4. We did not compare utility savings achieved by RAD-using PHAs with those 

using only EPCs. A possibility exists that RAD-related savings might be greater 

because RAD transformations are often holistic rehabilitation or construction of 

new units in which synergies among energy conservation investments can be 

realized. The comparison should be made to determine whether the RAD 

approach achieves as much if not more energy efficiency than the EPC 

approach.

5. We did not examine whether who paid utility bills between a PHA and the tenants 

made a difference in the PHA choosing to participate in the EPC program. HUD’s 

Resident-Paid Utility Incentive is intended to deal with this issue, but the extent to 

which PHAs have taken advantage of the incentive is unclear. Even with the 

incentive, however, PHA payment of utilities may provide greater inducement to 

undertake an EPC than when tenants pay the bills for individually metered 

utilities. This issue should be examined by HUD to determine whether PHAs with 

partial or full tenant utility payments are less inclined to undertake EPCs. 

1-1 

Chapter 1

Introduction

This study assesses the effectiveness and value of the U.S. Department of Housing and 

Urban Development’s (HUD’s) Energy Performance Contract (EPC) program by 

surveying public housing authorities (PHAs) and comparing trends in the utility 

consumption and the financial and physical condition of those PHAs that have 

implemented EPCs with those that have not. It also seeks to discover the extent to which 

implementation factors affect the rate at which EPCs are used by PHAs. HUD contracted 

with a team headed by LMI (the LMI team) for the comparative and implementation 

assessments. 

EPCs enable PHAs to secure private-sector financing to upgrade the energy efficiency 

of their facilities, utilizing the savings that result from these improvements to pay back 

the contractors. The availability of EPCs thus enables PHAs to invest more than would 

be possible through capital budgets, operating income, or HUD-provided subsidies 

alone. As a result, PHAs that make use of EPCs should be able to lower utility bills and 

improve the physical condition of their housing units. If the energy efficiency investments 

prove cost-effective, they should improve these PHAs’ financial condition as well. 

Several reasons exist, however, why a PHA may not use the EPC program to address 

energy efficiency investments. To gain approval from HUD to initiate an EPC, PHAs 

must go through several layers of review, requiring their staff to spend considerable time 

and effort. Further, not all prospective EPCs obtain positive returns, and there are other 

means whereby housing authorities might improve their utility, financial, and physical 

conditions, in particular, the Rental Assistance Demonstration (RAD) program,1 or by 

obtaining grants from state or local governmental authorities. Even if no outside 

financing is obtained, a PHA might invest in energy efficiency simply because it is costeffective to do so. Thus, it is not a foregone conclusion that PHAs that use EPCs will 

improve their utility consumption and other conditions at a greater rate than PHAs that 

do not.

This study followed four lines of investigation. First, the LMI team sent an online survey 

to more than 400 PHAs selected through a disproportionate stratified sampling strategy. 

One purpose was to obtain information from PHAs that have undertaken EPCs 

concerning their effectiveness in reducing utilities and in improving financial and physical 

conditions. The survey also obtained information from PHAs that have not undertaken

EPCs regarding improvements over time in the same three metrics. This latter group 

was broken into two parts: PHAs that at one point started but then did not complete the 

EPC process and those that had never begun an EPC. All survey respondents were 

asked their views on the implementation of HUD’s EPC program and how it might be 

improved. In all, 327 or more than 70 percent of those to whom the survey was sent 

responded.

1 RAD is a federal housing program administered by HUD that enables assisted rental housing to access private capital markets for 

purposes of development and rehabilitation. The program was initially authorized by Congress in 2012 as a demonstration program 

and since has expanded from a ceiling of 60,000 units to 455,000 units.
 

1-2 

Second, the online survey was followed by a set of telephone interviews with a subset of 

the responding PHAs stratified by size and geographic area that went into more depth 

concerning their experiences with EPCs or improving their utility consumption without 

the use of this program. In all, 27 PHAs were selected out of which 20 telephone 

interviews were conducted.

Third, various HUD databases containing information about utility consumption and the 

financial and physical condition of PHAs were examined. The LMI team obtained access 

to HUD’s Real Estate Assessment Center (REAC) databases that included the Financial 

Management Assessment Subsystem (FASS) and Physical Assessment Subsystem 

(PASS), and also to several years of Utility Expense Level (UEL) data, as submitted by 

PHAs on form 52722. This information enabled the LMI team to corroborate survey 

responses and to statistically test the efficacy of EPCs in improving public housing utility 

consumption and financial and physical conditions. 

A fourth approach looked to see whether PHAs prefer the RAD or EPC program as a 

means to finance improvements in upgrading the energy efficiency of their housing units. 

RAD enables PHAs to finance a broader range of improvements than the EPC program 

and has different implementation requirements. The number of housing units in the RAD 

program and number of EPC approvals and applications were compared to see to what 

extent RAD may have replaced EPCs for energy efficiency upgrades of PHA portfolios. 

The rest of this report is organized as follows: 

• Chapter 2, a brief background on the public housing EPC program.

• Chapter 3, the sampling method used to gather information on the relative 

effectiveness of HUD’s EPC program. 

• Chapter 4, how the survey was implemented.

• Chapter 5, survey results, including the effect of EPCs on utility consumption and 

what respondents reported with respect to EPC implementation.

• Chapter 6, results obtained from in-depth telephone conversations with a subset 

of PHA directors who responded to the survey.

• Chapter 7, analysis of HUD administrative data. These data were used to 

statistically test whether HUD’s EPC program affected PHA utility consumption 

as well as these entities’ financial and physical condition.

• Chapter 8, HUD’s RAD program and its impact on PHA choice with respect to 

financing various improvements to public housing.

• Chapter 9, conclusions from the study and recommendations.

• Appendix A, graphs showing the relative frequency of answers given to each of 

the survey questions.

• Appendix B, detailed notes on each of the 20 follow-up telephone interviews with 

PHAs. The notes summarize what each interviewee had to say, omitting only the 

identity of the individual PHA.

• Appendix C, discussion of methods used to deal with missing or improperly 

reported data.
Introduction

1-3 

• Appendix D, maps of HUD vs. U.S. census regions, and of PHA and sample 

populations. 

• Appendix E, list of abbreviations used in the study and their meanings.

2-1 

Chapter 2

Background

Energy performance contracts (also called energy service performance contracts or 

ESPCs) were first authorized by Congress in 1986. They were initially called shared 

energy service contracts and were authorized by the Consolidated Omnibus Budget 

Reconciliation Act of 1985.1 They were later identified as ESPCs in the Energy Policy 

Act of 1992.2 Such contracts are primarily aimed at enabling federal agencies to 

enhance the energy efficiency of their working facilities. ESPCs supply federal agencies 

with a source of funding for energy-saving purposes beyond annual appropriations. 

Under the act’s provisions, agencies can contract with private companies to furnish 

energy-saving equipment and the upfront money to finance it. Contractor investments, 

including a return on capital, are expected to be repaid from the resulting energy 

savings.

According to the Federal Energy Management Program (FEMP), nearly $5.2 billion has 

been invested within the ESPC program since its inception, resulting in cumulative 

energy cost savings of $12.3 billion. A wide variety of federal agencies have undertaken 

ESPC projects, including the General Services Administration, the Department of 

Defense, and the Department of Energy. FEMP guides and assists agencies in 

implementing ESPCs at their own facilities.

In HUD’s case, public housing properties are not regarded as federal buildings and

therefore are not covered by FEMP regulation. Instead, energy performance contracts in 

public housing were separately authorized under Section 118(a) of the Housing and 

Community Development Act of 19873 and implemented through revised Performance 

Funding Systems regulation within the Code of Federal Regulations (CFR).

4

HUD has a systematic approach to its EPC program (HUD, 2018c). A public housing 

authority prepares a request for proposals, conducts an energy audit, and creates a 

contract package with the contractor that will supply the energy investments. An initial 

review is conducted by a HUD field office, Energy Center, or contractor, and another 

level of review by a HUD panel, which examines the earlier review to be sure it is 

complete. The reviewers use technical and completeness review checklists to ensure 

that the project is technically sound and that it contains a complete set of required 

documents. Once approved, the project is authorized by a field Public Housing Director. 

From the applicant’s perspective, the key steps are gaining approval of the application 

by the field office or Energy Center, and confirmation that the review was complete from 

HUD’s review panel.

Of the approximately 3,000 PHAs in the United States, nearly 90 percent are small or 

very small with under 500 units. Medium (500–1,250 units), large (1,250–6,600 units), 

and very large PHAs (over 6,600 units) constitute the remaining 10 percent. Among 

1 Public Law (P.L.) 99–272, 100 Stat. 82, also known as COBRA 85.

2 102nd Congress H.R.776.ENR, abbreviated as EPAct 92.

3 P.L. 100-242, 101 Stat. 1815.

4 24 CFR 990, which applies specifically to HUD. 
 

2-2 

PHAs, a few hundred have undertaken EPCs. Nearly 50 percent of medium to large 

PHAs have undertaken them, but only about 4 percent of the smaller ones have done 

so. These relative proportions were taken into account in stratifying the survey sample 

as described in the next chapter.

Since 2012, PHAs have had access to another program, the Rental Assistance 

Demonstration, to tap private capital to enhance energy efficiency (along with other 

improvements) in public housing. This program converts PHA-owned properties to rental 

assistance under Section 8 of the United States Housing Act of 1937,

5 which furnishes 

project-based rental assistance or project-based voucher assistance over long-term 

contracts, typically 20 years. RAD provides incentives for PHAs to invest in energy 

efficiency because utility costs are frozen at pre-RAD levels, analogous to the “frozen 

rolling base” in EPCs. HUD’s long-term financial commitments to subsidize tenant rents 

serve as a credit enhancement to attract long-term debt and equity from the sale of LowIncome Housing Tax Credits (LIHTCs) in order to attract capital to upgrade facilities and 

reduce backlogs of unfunded capital improvements. In Chapter 8, the effect of RAD on 

HUD’s EPC program is examined to see whether it significantly changed PHA 

preferences in obtaining private finance to upgrade housing units.

5 Enacted September 1, 1937 (P.L. 75-412).
3-1 

Chapter 3

Survey Sample

Research Population

The chief method of gathering data on the relative performance of Energy Performance 

Contract-using and non-EPC-using public housing authorities with regards to utility, 

financial, and physical conditions was a survey of PHAs. The population under study 

consisted of three groups: Group 1—executed at least one EPC; Group 2—started the 

process but did not complete an EPC; and Group 3—never executed an EPC. For 

purposes of comparing relative performance, we mainly looked at Groups 1 and 3. There 

were 2,971 PHAs in all. Of those, 237 had implemented an EPC by fiscal year (FY) 2015

or earlier1 and 2,734 PHAs had not (we refer to this group as non-EPC).

Sample Design and Process

We categorized sizes of the PHAs as small (very small/small), medium, and large 

(large/very large). We categorized regions according to U.S. Census regions: South, 

West/Midwest, and Northeast. Because the West region has smaller subgroup counts, 

we combined the West and Midwest regions into a single category for purposes of 

getting a nationwide representation. We also compared the three census regions with

the 10 HUD regions to ensure consistent mapping. Exhibit 3-1 shows the mapping of the 

10 HUD regions to the census regions for each state. In appendix D we show the two 

regional breakdowns in map form. Appendix D also contains maps showing the 

distribution of the population of PHAs among the states and the number included in our 

sample in each state. 

Exhibit 3-1. Mapping of HUD Regions to Census Regions

HUD region Area name Census region

New England Connecticut Northeast

New England Maine Northeast

New England Massachusetts Northeast

New England New Hampshire Northeast

New England Rhode Island Northeast

New England Vermont Northeast

New York/New Jersey New Jersey Northeast

New York/New Jersey New York Northeast

Mid Atlantic Washington, DC South

Mid Atlantic Delaware South

Mid Atlantic Maryland South

1 We chose FY 2015 as the cutoff because we wanted to compare performance after an EPC had been implemented with before 

implementation and needed at least a few years of post-EPC utility consumption to do so. Choosing FY 2015 as the cutoff meant 

that, at a minimum, we would have 3 years of post-EPC implementation data. 
 

3-2 

Exhibit 3-1. Mapping of HUD Regions to Census Regions

HUD region Area name Census region

Mid Atlantic Pennsylvania Northeast

Mid Atlantic Virginia South

Mid Atlantic West Virginia South

Southeast Caribbean Puerto Rico South

Southeast Caribbean Virgin Islands South

Southeast Caribbean Alabama South

Southeast Caribbean Florida South

Southeast Caribbean Georgia South

Southeast Caribbean Kentucky South

Southeast Caribbean Mississippi South

Southeast Caribbean North Carolina South

Southeast Caribbean South Carolina South

Southeast Caribbean Tennessee South

Midwest Illinois West/Midwest

Midwest Indiana West/Midwest

Midwest Michigan West/Midwest

Midwest Minnesota West/Midwest

Midwest Ohio West/Midwest

Midwest Wisconsin West/Midwest

Southwest Arkansas South

Southwest Louisiana South

Southwest New Mexico West/Midwest

Southwest Oklahoma South

Southwest Texas South

Great Plains Iowa West/Midwest

Great Plains Kansas West/Midwest

Great Plains Missouri West/Midwest

Great Plains Nebraska West/Midwest

Rocky Mountain Colorado West/Midwest

Rocky Mountain Montana West/Midwest

Rocky Mountain North Dakota West/Midwest

Rocky Mountain South Dakota West/Midwest

Rocky Mountain Utah West/Midwest

Rocky Mountain Wyoming West/Midwest

Pacific/Hawaii Arizona West/Midwest

Pacific/Hawaii California West/Midwest

Pacific/Hawaii Hawaii West/Midwest

Pacific/Hawaii Nevada West/Midwest

Northwest/Alaska Alaska West/Midwest

Northwest/Alaska Idaho West/Midwest
Survey Sample

3-3 

Exhibit 3-1. Mapping of HUD Regions to Census Regions

HUD region Area name Census region

Northwest/Alaska Oregon West/Midwest

Northwest/Alaska Washington West/Midwest

The population counts for the three size categories are shown in exhibit 3-2. The size 

distribution of the population is skewed, with fewer numbers of large PHAs than medium 

and small PHAs. Large PHAs, however, account for most of the units affected by EPCs. 

For any moderately sized sample, an equal probability sample design would include

extremely small numbers of larger PHAs. Therefore, to allow for separate analysis by 

size, we selected a fairly large sample size per size-band. This method was achieved by 

systematically increasing the sampling fraction as the size of the PHA increases.

Exhibit 3-2. Group Population Counts

 Group 1 Group 2 Group 3

Small South 24 10 1,308

West/Midwest 33 15 961

Northeast 43 7 251

Subtotal 100 32 2,520

Medium South 26 2 56

West/Midwest 21 5 35

Northeast 27 3 29

Sub-total 74 10 120

Large South 15 7 19

West/Midwest 18 2 16

Northeast 30 0 8 

Subtotal 63 9 43

Total 237 51 2,683

Group 1: For the EPC population, we employed a disproportionate stratified sampling 

strategy for the three size categories defined in exhibit 3-2. Specifically, we applied an 

incremental sampling fraction for each size category from small to large PHAs. Large 

PHAs were selected with certainty; that is, the sampling fraction was one, medium was 

0.8 and small 0.6. In addition, the sample is stratified by region, although within each 

size category the sampling fraction is kept roughly constant and the sub-samples 

selected proportional to corresponding population counts (exhibit 3-3). 
 

3-4 

Exhibit 3-3. EPC Sampling Strategy

 EPC population EPC sample Sampling fraction

Small South 24 16 0.6

West/Midwest 33 21 0.6

Northeast 43 28 0.6

Subtotal 100 65 0.6

Medium South 26 21 0.8

West/Midwest 21 17 0.8

Northeast 27 22 0.8

Sub-total 74 60 0.8

Large South 15 15 1.0

West/Midwest 18 18 1.0

Northeast 30 30 1.0

Subtotal 63 63 1.0

Total 237 188

Group 2: Group 2 members (PHAs that had initiated but did not complete an EPC) were 

selected with certainty as there are only 51 such PHAs in the population.

Group 3: For the non-EPC population, we matched the subgroups to make this group as 

similar as possible to the EPC population. Large PHAs were selected with certainty. 

There are only 43 large PHAs in Group 3 compared with 63 in Group 1. The sampling 

fraction for medium PHAs was 0.57 and that for small only 0.03, because there are 

thousands of small PHAs that have not conducted EPCs. We increased the small and 

medium size-bands in Group 3 to compensate for the smaller number of large PHAs as 

compared with Group 1 (exhibit 3-4). 

Exhibit 3-4. Non-EPC Sampling Strategy

 

Non-EPC 

population

Non-EPC 

sample 

Sampling 

fraction 

Small South 1,308 20 0.02

West/Midwest 961 25 0.03

Northeast 251 32 0.13

Subtotal 2,520 77 0.03

Medium South 56 24 0.43

West/Midwest 35 20 0.57

Northeast 29 24 0.83

Subtotal 120 68 0.57

Large South 19 19 1.00

West/Midwest 16 16 1.00

Northeast 8 8 1.00

Subtotal 43 43 1.00

Total 2,683 188
Survey Sample

3-5 

We expected (and obtained) about a 70-percent response rate to our nationwide survey 

of PHAs. Our aim was to obtain Group 1 and Group 3 samples of about 132 responses 

each, sufficient to establish statistical significance if differences were found between the 

two groups in utility, financial, or physical condition performance.

4-1 

Chapter 4

Survey Implementation1

Survey Response Rate 

Following the Office of Management and Budget approval of the survey instrument,

2 it 

was sent to 427 public housing authorities’ contacts. As indicated in Chapter 3, these 

PHAs were divided into three groups, 188 to those that completed an Energy 

Performance Contract, 51 to those that had started but not finished an EPC, and 188 to 

those that had never undertaken an EPC. Initial communication to recipients took the 

form of a note from HUD staff asking for cooperation with the survey team. Following 

that note, the survey was sent out on August 24, 2018, and closed as of September 19, 

2018 (4 weeks). The survey was followed by several reminders to complete it by a fixed 

date. These reminders were sent every few days beginning about a week after the 

survey was initially sent and ending a day or two before the survey period closed.

In some cases, the email address given for a PHA contact did not work and the survey 

bounced back. In many of those instances, however, an alternate email address was 

secured and the survey was sent to the alternate recipient. In six cases, however, no 

alternate address was obtained, the survey was treated as not sent, and six was 

subtracted from the total in calculating the response rates. The revised total, therefore,

was 421.

Two types of responses to the survey were received: complete and partial. Although 

ongoing efforts were made to explain the survey to respondents and to encourage 

completion, not all survey responses had been completely filled out by the time the 

survey period ended. In all, 327 responses were received, of which 280 were fully 

completed.

Exhibit 4-1 shows the cumulative response rate over time of those that responded either 

partially or fully. Fourteen percent responded soon after the initial request and, during

the next 4 weeks, the rate rose to 78 percent. This rate exceeded the rate expected by 

LMI in its initial formulation of a project plan (70 percent).

1 The survey questions and the relative frequencies of answers received for each are shown in appendix A. 

2 Reginfo.gov, ICR Reference No: 201711-2528-001, https://www.reginfo.gov/public/do/PRAViewICR?ref_nbr=201711-2528-001. 
 

4-2 

Exhibit 4-1. Full and Partial Response Rate

In exhibit 4-2, the response rate of only the 280 PHAs that filled the survey out 

completely is shown. The rate begins at around 13 percent and then rises over time, 

finally getting to 67 percent after about 4 weeks. Although the LMI team would have 

preferred that all responses were completely filled out, a 67 percent rate is still quite 

substantial.

Exhibit 4-2. Completed Survey Response Rate
Survey Implementation

4-3 

The rate of response was separated into regions as well. The regional response rates 

are shown in exhibit 4-3. This table shows fairly uniform regional response rates for fully 

plus partially completed surveys, but some discrepancy among regions for only fully 

completed surveys.

Exhibit 4-3. Survey Responses by Region

Region

Fully + partially 

completed surveys 

(#) 

Fully + partially 

completed surveys 

(%)

Fully completed 

surveys only 

(#) 

Fully completed 

surveys only

(%)

Midwest 76 81.7 70 75.3

Northeast 114 74.5 94 61.4

South 105 78.9 92 69.2

West 32 76.2 24 57.1

Total 327 77.7 280 66.5

Corrections to the Data3

Blank or Incorrect Responses

Although the high response rate to the survey was gratifying, not all the responses were 

correctly filled out and a few had to be eliminated. For example, the 327 responses 

included 19 in which the respondent entered the survey website but failed to fill out any 

questions at all. We cannot be sure why this situation occurred, but, as there was no 

useful data from these 19 responses, they were eliminated from further consideration, 

leaving 308 or about a 73-percent response rate.

Five other respondents appeared to misunderstand what they were expected to do and 

ended up answering the wrong set of questions. In other words, they classified 

themselves in one fashion but answered a different set of questions than that 

classification required. These five were therefore eliminated as well, leaving 303 

successful responses. 

Discrepancies in Respondent Group Category Designations

A fairly large number of respondents classified themselves differently than the LMI team 

had based on data provided by HUD. Exhibit 4-4, for example, shows that 21 PHAs 

categorized themselves as having executed an EPC whereas HUD data showed them 

as having started but not completed an EPC. In addition, 10 PHAs said they had 

executed an EPC whereas HUD data indicated they had not. 

3 The steps taken to correct the survey data are discussed in more detail in appendix C. 
 

4-4 

Exhibit 4-4. PHA Classifies Itself as Group 1 Whereas HUD Information 

Classifies the PHA as Group 2 or 3

Classification discrepancy Number

PHA—Group 1, HUD data—Group 2 21

PHA—Group 1, HUD data—Group 3 10

Total 31

It is possible that project timing could explain some of this discrepancy. For example, if a 

PHA started an EPC, did not complete it at the time but later completed it, HUD 

information might have classified it within Group 2 whereas the PHA would classify itself 

as Group 1. It is also possible that the PHA undertook some other, non-EPC energy 

efficiency project which it then mistakenly classified.

Exhibit 4-5 shows the opposite type of discrepancy, namely where HUD information 

indicates a PHA has done an EPC whereas the PHA designated itself as in Group 2 or 

Group 3. Some of this discrepancy might be explained by instances where there has 

been turnover in PHA management and the present management is unaware that an 

EPC was done previously.

Exhibit 4-5. HUD Data Classifies PHA as Group 1 Whereas PHA

Classifies Itself as Group 2 or 3

Classification discrepancy Number

HUD—Group 1, PHA—Group 2 5 

HUD—Group 1, PHA—Group 3 7 

Total 12

In exhibit 4-6, HUD data and individual PHAs agree that no EPC was done, but, in 21

cases, HUD data indicate that the PHA never did an EPC whereas the PHA indicates it 

started but stopped the process. We cannot be sure what a PHA meant when it 

indicated it started the process; possibly some of the starts involved little more than an 

initial review that did not go very far. In two cases, HUD data indicated a PHA had 

started but not completed an EPC whereas the PHA said it never started one. This

discrepancy could result from a PHA thinking it never looked seriously at an EPC 

whereas HUD classified it as having started the process, or by personnel turnover at a 

PHA, with new leadership unaware that previous leadership had started the EPC 

process.

Exhibit 4-6. HUD and PHA Disagree on Whether the PHA is in Group 2 or 3

Classification discrepancy Number

HUD—Group 3, PHA—Group 2 21

HUD—Group 2, PHA—Group 3 2 

Total 23
Survey Implementation

4-5 

In all, more PHAs classified themselves as in Groups 1 or 2 than HUD data. Assuming 

most of these PHAs did, in fact, engage with HUD’s EPC process, more PHAs have 

been willing to do so than previously thought.

Implications for Survey Response Analysis 

Because we wanted to see whether an EPC made a meaningful difference in energy 

performance, it was important that we make appropriate corrections to the survey data. 

As already mentioned, six instances in which we were unable to secure correct email 

addresses were eliminated. We also eliminated the 19 responses in which the 

respondents entered the survey website but did not answer any questions, and five more 

where the respondents appeared to answer the wrong set of questions.

Concerning the discrepancies between how PHAs classified themselves and how the 

LMI team performed based on HUD-supplied data, we generally took the view that the 

PHA would have the best knowledge as to whether it considered or completed an EPC. 

In most cases, therefore, we accepted the PHA’s classification and assumed it had 

answered the appropriate set of survey questions.

After detailed review with the Energy Center, however, we eliminated nine more 

responses from consideration as it appeared that the center had conclusive information 

that a PHA had, in fact, completed an EPC, whereas the PHA said it had not. Elimination 

of these nine PHAs reduced our sample further, to 294 PHAs.

Adjustment of Sample Sizes

Because we eliminated responses on various grounds, and in 58 cases accepted PHA 

classifications of whether they belonged to Groups 1, 2, or 3 as opposed to our original 

classification based on HUD data, we had to make adjustments to our group sample 

sizes. Once this modification was done, we ended with 160 responses from Group 1, 37 

from Group 2, and 97 from Group 3. Group 1 and Group 3 samples were used to test 

whether the energy performance and financial and physical condition of PHAs that

conducted EPCs were significantly different from those that did not.

In exhibit 4-7, we show the three groups separated into small, medium, and large PHA 

subgroups. For Group 1, 54 percent of total responses, we received about the same 

number of useful responses from each of the three subgroups, whereas for Groups 2 (13

percent of total responses) and 3 (33 percent of total responses), we received 

successively more useful responses as PHA size declined. For statistical testing 

purposes, however, we did not distinguish among the size classes of the PHAs.
 

4-6 

Exhibit 4-7. Final Numbers of Survey Responses by Group and PHA Size
5-1 

Chapter 5

Survey Analysis

Statistical Weighting

Our approach to testing the efficacy of HUD’s Energy Performance Contract program 

consisted largely of comparing performance measures between the EPC and non-EPC 

groups (Group 1 and Group 3), using summary statistics such as means/averages and 

proportions. In probability-based sample designs, however, the collected data must

account for selection probabilities and be adjusted for differences in proportions among 

the different strata or subgroups. We employed the following steps to weight the survey 

data collected and calibrate the data to population totals.

1. Base weights: Our sample design was stratified in such a way that the selection 

probabilities were not equal among the different strata. For example, small South 

public housing authorities in Group 1 had a sampling probability of 0.6 whereas 

small South PHAs in Group 3 had a selection probability of 0.013. To account for 

unequal selection probabilities, we calculate base weights by taking the inverse 

of selection probabilities.

Base weight h = NNh

nnh, 

Where Nh is the population count of stratum h and nh is the sample size in 

stratum h.

2. Adjust for non-responses: As noted in Chapter 4, not all PHAs to whom the 

survey was sent responded. We calculated response rate (RR) adjustment 

factors by taking the inverse of the response rate for each stratum as shown in 

the following formula: 

RR weight h = nnh

rrh, 

Where nh is the sample size of stratum h and rh is the number of responses 

collected in stratum h.

3. Applying both weights together, we arrived at a combined weight for each 

stratum h using the formula:

Final weight h = base weight h x RR weight h.

This final weight calibrates all responses to known population counts and keeps the 

response proportions of each stratum the same as in the population.

The weighted data obtained using the steps described previously are only used for 

calculating weighted means, weighted standard deviations, and weighted standard 

errors when comparing Groups 1 and 3 and performing statistical hypotheses testing. 

Other reports, frequencies, cross-tabulations, charts, and tables are presented using the 

raw, unweighted data.
 

5-2 

Statistical Hypotheses Testing—Comparing Group 1 with

Group 3

The survey questions included in the statistical analyses for group comparisons are 

shown in exhibit 5-1. These inquiries include questions on utility, financial, and physical 

condition performance.

Exhibit 5-1. Survey Questions Relating to Utility, Financial, and

Physical Condition

Group 1: EPC Group 3: Alternative investments

Group 3: No energy 

investments

Q9–Q11. On average, by how much 

did utility consumption change due to 

the EPCs you implemented? 

Q40–Q42. On average, by how 

much did utility consumption 

change due to the utility 

conservation investments you 

made? 

Q12–Q14. On average, by how 

much did the following utility 

expenses change due to the EPCs 

you implemented?

Q43–Q45. On average, by how 

much did utility expenses change 

due to the utility conservation 

investments you made? 

Q21. Overall, how did the EPCs you 

executed affect the financial 

condition of your PHA?

Q47. How did the utility 

conservation investments affect the 

overall financial condition of your 

PHA? 

Q49. Over the past 5 years, 

would you say that the 

financial condition of your PHA 

has shown: (measure of extent 

of improvement)? 

Q22. How did the EPCs affect the 

overall physical condition of the 

included properties?

Q48. How did the utility 

conservation investments affect the 

overall physical condition of the 

properties in your PHA? 

Q50. Over the past 5 years, 

would you say that the 

physical condition of the 

properties within your PHA 

has shown: (measure of extent 

of improvement)? 

Development of Ratings and Averages

For each of the questions specified in exhibit 5-1, we needed to standardize the ratings 

across both of the groups to make meaningful comparisons. For that purpose, numeric 

ratings were created. These ratings are shown in exhibit 5-2. If a respondent indicated it 

experienced both an increase and a decrease of consumption or expenses for any of the 

utility sources, however, that response was removed from the data set and the test.

Exhibit 5-2. Ratings Attached to Answers to Utility-Related Questions

Calculation of Weighted Average Scores

After standardizing the rating scores as indicated, we compared ratings of Group 1 

PHAs with Group 3 PHAs using weighted averages for consumption and expense for 

each utility (electricity, natural gas, fuel oil, and water) and for financial and physical 
Survey Analysis

5-3 

conditions. The weighted average ratings were calculated using the following generic 

formula: 

Weighted average = ∑ ww(ii)∗rrrrrriinnrr(ii) nn

ii=1

∑ ww(ii) nn

ii=1

, 

Where w(i) = the final weight obtained after the weighting process, rating(i) = the 

corresponding rating score calculated for the measure, and n = the number of 

responses.

We illustrate with an example calculation for the weighted average of electricity 

consumption for Group 1. The related question for this example is

Q9. On average, by how much did electricity consumption change due to the 

EPC(s) you implemented?

Here, n = number of EPC PHAs that responded to Q9 = 111.

Numerator = sum (weights ÷ electricity consumption rating) for 111 respondents = 375.63.

Denominator = sum of all weights of the 111 respondents = 173.16.

Weighted average = numerator/denominator = 375.63/173.16 = 2.1692.

Similarly, weighted averages were calculated for all the different measures that were 

compared across Groups 1 and 3. The detailed averages, standard deviations, and 

standard errors are shown in exhibit 5-3. Among the weighted averages for utilities, a 

lower number means a better score; that is, a bigger reduction in utility use or payments. 

For financial and physical condition, a bigger number indicates a better score; that is, a 

bigger improvement over time. 

Exhibit 5-3. Weighted Averages, Standard Deviations, and Standard Errors

Group

Number of 

responses (n)

Weighted 

average/mean

Weighted std. 

deviation

Std. error mean 

s/sqrt(n) 

Electricity 

consumption

1 111 2.1692 1.63753 0.15543

3 37 2.6243 5.46642 0.89867

Electricity 

expense

1 111 2.4410 1.87246 0.17773

3 38 2.6286 5.36416 0.87018

Natural gas 

consumption

1 105 2.0652 1.63492 0.15955

3 22 2.7579 5.07379 1.08174

Natural gas 

expense

1 105 2.2909 1.77449 0.17317

3 21 2.8741 3.20220 0.69878

Fuel oil 

consumption

1 15 2.2198 2.04196 0.52723

3 7 2.9948 4.24903 1.60598

Fuel oil expense 1 13 2.1455 2.04764 0.56791

3 7 3.1496 2.75227 1.04026

Water 

consumption

1 114 1.7196 1.49936 0.14043

3 30 2.5306 5.15194 0.94061

Water expense 1 111 2.1115 1.89231 0.17961

3 29 2.5098 4.94338 0.91796
 

5-4 

Exhibit 5-3. Weighted Averages, Standard Deviations, and Standard Errors

Group

Number of 

responses (n)

Weighted 

average/mean

Weighted std. 

deviation

Std. error mean 

s/sqrt(n) 

Financial 

condition

1 144 3.5529 1.13746 0.09479

3 84 3.3707 4.51324 0.49243

Physical 

condition

1 148 4.0115 0.72607 0.5968

3 84 3.7708 3.96467 0.43258

Hypotheses Testing

We tested whether there were statistically significant differences in performance 

between Groups 1 and 3 for each of the consumption, expense, physical, and financial 

measures. The null hypothesis in each case was that “there is no difference between the 

averages/means of a given measure between the EPC group and the non-EPC group.” 

The test, therefore, was whether the null hypothesis was rejected, that is, whether there 

was a statistically significant difference between the group averages. 

Exhibits 5-4 and 5-5 show the results of the hypothesis testing. The measures with an 

asterisk (*) indicate that the null hypothesis that there was no difference between the 

groups was rejected at a 95 percent confidence level, meaning that there was a 

difference and that the chances that this difference resulted from random factors are no 

more than 5 percent.1

Exhibit 5-4. Group 1 and Group 3 Differences in Average Utility Scores

(* indicates statistical significance at the 95-percent level)

Exhibit 5-4 indicates that among the eight measures, only three showed statistically 

significant differences at the 95-percent level: natural gas consumption, natural gas 

expense, and water consumption. The other differences between weighted averages did 

not achieve that level of statistical confidence. The Group 1 respondents, however,

scored lower in water expense, electricity consumption, electricity expenditures, fuel oil 

consumption, and fuel oil expense. This result means that the implementation of an EPC 

1 More generally, results indicating statistical significance at a 95 percent (or 90 percent) confidence level means that we are 

95 percent (90 percent) confident that there is a true difference in performance between EPC PHAs and non-EPC PHAs. If a result 

is not statistically significant at a 95 percent (90 percent) level, then the chances that whatever difference occurred resulted from 

random factors is greater than 5 percent (10 percent). 
Survey Analysis

5-5 

made a positive difference in these utilities, although not one we could infer was 

statistically significant at the 95-percent level.

Exhibit 5-5. Differences Between Average Financial and Physical Ratings

(* indicates statistical significance at the 95-percent level)

Exhibit 5-5 shows average scores for financial and physical conditions. For those 

categories, a higher number represents a better outcome. Although the numbers 

indicate that the financial condition of those PHAs that executed an EPC improved by 

more than such condition for those that did not, the difference is not statistically 

significant. At the same time, there was a statistically significant difference in the 

improvement of the physical condition between those that executed an EPC and those 

that did not. This difference may have arisen from the investments in energy-efficient 

equipment, or from financial returns from the energy investments which then could be 

used for other upgrades, or from both.

Within Group 1 Differences—Units Subject to an EPC vs. 

Units Not Subject to an EPC

We also examined Group 1 survey responses to see whether respondents thought there 

were differences in performance between units within PHAs that had implemented an 

EPC and those that had not. We first examined whether a Group 1 respondent, having 

completed at least one EPC, would consider doing another. The responses are shown in 

exhibit 5-6. Approximately 60 percent of PHAs that had completed at least one EPC 

indicated they were favorably disposed toward doing another. Because at least some of 

the other 40 percent could already have accomplished all the energy investments they 

considered cost-effective, we consider this convincing evidence that PHAs that had gone 

through the EPC process at least once found it useful.

Exhibit 5-6. Percentage of Group 1 Respondents Who Would Do Another EPC

We then asked those that had done an EPC to compare utility, financial, and physical 

condition performance at those units that had an EPC with such performance at units 
 

5-6 

that had not. The questions and distributions of responses are shown in exhibits 5-7, 5-

8, and 5-9. 

Exhibit 5-7. Survey Responses on Utility Performance at EPC vs. Non-EPC Units

Exhibit 5-8. Survey Responses on Financial Condition at EPC vs. Non-EPC Unitsa

a The response options for this question included “Somewhat worse at the units subject to the EPC” but no respondent chose 

that answer.

Exhibit 5-9. Survey Responses on Physical Condition at EPC vs. Non-EPC Unitsa

a Response options for physical condition at EPC units vs. non-EPC included “a lot worse” and “somewhat worse” but no 

respondent chose either of those two answers.

Nearly 70 percent of the Group 1 respondents thought that utility performance and 

financial and physical conditions had improved more at their EPC units than the nonEPC units. These findings are consistent with our statistical tests of differences in 

performance between PHAs that executed EPCs and those that did not, and provide a 

positive rationale for why Group 1 respondents would express willingness to do another 

EPC.

Σ = 66.67%

Σ = 69.49%

Σ = 67.86%
Survey Analysis

5-7 

Survey Evidence Concerning Why PHAs Did Not 

Undertake EPCs

Group 3 and Group 2 PHAs were asked why they had not undertaken an EPC or why, if 

they had started the process, they did not complete it. Exhibit 5-10 shows the responses 

received from those that had never done an EPC.

Exhibit 5-10. Why PHAs Chose Not to Do an EPC

As shown in the figure, the principal reason given (39 percent) for not doing an EPC was 

a matter of economics; the EPC would not be cost-effective, at least from the PHA’s 

point of view. We were unable to tell from this answer whether it occurred because, 

despite HUD’s Resident-Paid Utilities Incentive program (RPU), the PHA would not 

receive enough of the savings, or whether it simply meant that an EPC would not pay for 

itself regardless of who paid the utilities. This question might be investigated further.

2

Another important reason was the option to enter the Rental Assistance Demonstration

program instead. The scope and financial basis of RAD are different from those of an 

EPC and appeared to appeal to a significant proportion of those answering this question.

Those that had started but not completed an EPC responded as shown in exhibit 5-11

when asked why they did so. The figure indicates that the combined reasons of 

insufficient staff, paperwork, and expense cover more than one-half of those that

stopped during the process. These factors may be more significant mainly to smaller 

PHAs, for whom the management of an EPC might constitute a significant outlay.

2 HUD’s RPU is designed to overcome any disincentive to PHA investment in energy efficiency from residents paying part or all the 

utilities. Data supplied by the Energy Center suggests that around 40 percent of PHAs that undertake EPCs take advantage of the

incentive program (40 of 105 during the 6-year period 2012–2017 with 3 of the 105 indeterminate), but gives no indication whether 

those that did not take advantage had resident paid utilities or not. Recent HUD-supplied data indicates, however, that, as of 2018, 

about 73 percent of households within one or another form of public housing receive utility allowances. This data suggests, although 

it does not prove, that some PHAs with tenant paid utilities are not taking advantage of the RPU program. 
 

5-8 

Exhibit 5-11. Why PHAs Began but Did Not Complete the EPC Process

How to Improve HUD’s EPC Process 

An important aspect of this project was to examine the implementation of HUD’s EPC 

process to find ways it could be improved. All three groups of PHAs were asked to 

address this question directly. Respondents could check multiple answers if they chose, 

and therefore the totals are more than the total number of responses to the survey. 

Exhibit 5-12 shows the distribution of responses.

Exhibit 5-12. Ways to Improve HUD’s EPC Process Suggested by PHAs

As seen in the figure, the principal answer given was to increase operating subsidies to 

PHAs undertaking an EPC. The fourth highest response also indicated a desire for direct 

payment, whereas the third asked for more assistance in kind (that is, technical help). 

The second most prevalent answer asked HUD to simplify its EPC process, although we 

did not ask respondents to identify ways this simplification should be done and few 

offered specifics. A few of our telephone interview contacts indicated that one way PHAs 

overcome the administrative requirements is to let their contractor help because some of 

the contractors have experience with the EPC process. Contractors charge for this 

service, however, altering the economics of the project from what they otherwise would 

be.
6-1 

Chapter 6

Telephone Follow-up Interviews

Introduction

To gain further insight into public housing authorities’ views on HUD’s Energy 

Performance Contract program, the team interviewed a number of the participants by 

telephone after the written survey was completed. Some 27 PHAs were selected to be 

interviewed, consisting of 9 from each group. The three groups were further divided into 

three each of small, medium, and large PHAs to sample opinions from as many different 

perspectives as possible within the limited sample.

Several efforts were made to contact the 27 PHAs to secure their participation in the 

follow-up telephone interviews. An initial note requesting participation was followed by 

two reminder notes plus a set of personal emails to PHA directors who had worked with 

one of the LMI team members. These efforts ultimately resulted in interviews with 20 of 

the 27 initially contacted, or a response rate of 74 percent.

Representation by Group and Size

The 20 interviewees consisted of 8 from Group 1, 6 from Group 2, and 6 from Group 3. 

Thus, 40 percent had completed at least one EPC but the other 60 percent had not.

We interviewed representatives of eight small, five medium, and seven large PHAs. In 

exhibit 6-1, we show how the interview sample broke down between the two 

classifications (that is, by group and by size). Generally, within a total sample of 20, we 

were able to obtain a reasonable number of perspectives from each group and from 

each size category of PHA. 

Exhibit 6-1. Breakdown of Interview Sample by Group and PHA Size

Group 1 Group 2 Group 3 Total

Small 3 2 3 8 

Medium 3 1 1 5 

Large 2 3 2 7 

Total 8 6 6 20

Survey Results 

Although each PHA had something unique to offer concerning its experience (or nonexperience) with HUD’s EPC program, a number of themes emerged from the 

interviews. These issues are briefly enumerated below.

1. A great deal of Rental Assistance Demonstration participation is taking place 

among PHAs whether or not they have previously completed an EPC.
 

6-2 

RAD appears to have become most PHAs’ option of choice with regards to 

investment in facility upgrades. Several reasons were given, but the main one is 

that RAD enables a greater scope of investment in property upgrades. Because 

many PHAs see capital funding needs as reaching well beyond energy-related 

investments, this feature of RAD is especially attractive.

A second reason is that funding for RAD through the capital fund has been 

somewhat more stable than past capital funding. This consistency has increased 

the confidence of lenders and made RAD an attractive investment for them.

In addition, some PHAs see RAD as simpler to navigate than the EPC program, 

however, other PHAs said that the RAD program can be difficult to negotiate.

2. The EPC and RAD programs are considered by PHAs to be alternatives. 

Several PHAs remarked that they view EPCs and RAD as alternatives, not 

programs that should be pursued simultaneously. They see the programs as 

different means of raising private capital, with RAD the better choice if capital 

needs go beyond energy investment.

3. Energy savings are typically incorporated into any improvement project.

A number of PHAs remarked that they routinely look at potential energy savings 

from new investments whether or not they are EPC- or RAD-related. If they have 

not obtained funds from either of these programs, then they use some 

combination of operating and capital funds, state grants, and utility grants to 

make the needed energy investments.

4. If utilities are tenant paid, PHAs may not have the incentive to pursue the EPC 

program.

In cases where utilities were entirely owner paid, PHAs often found that EPC or 

RAD energy investments would pay off. If utilities were partially or entirely tenant 

paid, however, PHAs were less likely to undertake EPC investments although, if 

still made, the savings were popular with tenants.1

5. In several cases, the payoff from an EPC came mainly from water savings.

In several cases where the PHA felt its EPC investment had clearly paid off, the 

savings had come mainly from reduced water bills, not energy bills. It could be 

that energy savings had already been wrung from the properties prior to the 

EPC, that tenants increased their electricity usage over time, or that water rates 

had risen more steeply in recent years, but, in any case, the people interviewed 

were satisfied with their EPCs because the water savings had made the 

investments cost-effective.

6. PHAs are generally open to any program or funding source as a means to 

improve their housing stock.

Several PHA directors said they are open to any source of funding that will help 

them improve the physical condition of their properties. This funding includes 

EPCs, RAD, state and local funding sources, or utility programs that offer upfront 

capital. From this perspective, if an EPC offers the best deal to fund 

1 As indicated in Note 2, Chapter 5, p. 5–7, HUD’s Resident-Paid Utilities Incentive (RPU) program is designed to maintain PHA 

incentive to invest in energy efficiency even if utilities are tenant paid. We were unable to tell from existing data, however, the extent 

to which PHAs with units where utilities are at least partly tenant paid take advantage of this program. 
Telephone Follow-up Interviews

6-3 

improvements, they are open to it, but if other programs, typically RAD, offer 

better possibilities, they will consider those first.

Interviewee Recommendations

Because an important aim of this study was to identify ways in which HUD might be able 

to improve the implementation of its EPC program, interviewees were asked for their 

recommendations. Such recommendations already had been sought from the written 

survey, but the telephone follow-up interviews enabled more in-depth discussion. We 

emphasize that EPC program implementation recommendations were not necessarily 

uniform among PHAs; some recommended one thing, some another. The following, 

however, are the most common recommendations among the 20 PHAs we interviewed.

1. Better integrate and coordinate the RAD and EPC programs to help PHAs select 

the better program for their situation.

A number of PHAs felt that they could use more assistance in understanding the 

relative advantages of RAD vs. EPCs as well as the complexities of substituting 

one for the other or doing both simultaneously. Although they understood the 

basics of both programs, they were unsure how each best fit into their particular 

situation and recommended that HUD furnish more assistance in helping them 

navigate the choices.2

2. Help PHAs decide if and how the RAD and EPC programs can be used together.

This recommendation follows from the first. PHAs are open to all sources of 

funding, and if an EPC plus RAD offers more funding at better terms in a 

particular instance, the PHA would want to take advantage. The complexities of 

doing both are difficult to work out, however, and PHAs would like help from HUD 

in considering such an option.

3. Increase communication about the EPC program, sharing success stories, and 

creating a network of PHAs to share experiences and supply expertise.

A few PHAs felt that other PHAs either do not fully understand the EPC program 

or have heard negative stories about it that make them shy of initiating their own 

effort. Those holding such views recommend that HUD furnish more positive 

publicity for the program and also connect PHAs that have completed EPCs with 

those that have not for advice and guidance.

4. Train PHAs and field office staff to better understand the EPC program, including

a. the procurement process and how to negotiate an EPC,

b. how to manage a contractor,

c. an explanation of how PHAs benefit from the savings, and

d. how to educate and engage tenants and maintenance staff upfront.3

2 HUD offers PHAs a document entitled “The Rental Assistance Demonstration (RAD) and Energy Performance Contracts (EPCs)” 

at https://www.hud.gov/sites/dfiles/PIH/documents/RAD_EPC_FAQs.pdf. It is unclear to what extent interviewees are aware of the 

document.

3 Although some interviewees offered this recommendation, HUD did in fact offer multiple 1-day and 2-day EPC training sessions to 

PHAs between 2013 and 2015. These training sessions were held around the United States. In addition, in 2018, HUD sponsored a 

number of 3-day training sessions for PHAs focused on how to evaluate EPC savings. 
 

6-4 

Some PHA directors felt that a number of others, mainly but not only smaller 

PHAs, were either unaware of how to conduct an EPC or had no idea how to 

work with the EPC contractor to obtain maximum benefits. They particularly 

stressed that an EPC contractor needs to be closely managed to be sure that the 

PHA understands what the contractor is doing and how the financial benefits are 

being shared.

5. Streamline the application process and supply more technical assistance, 

especially for smaller PHAs.

A number of PHAs offered this pair of recommendations. Streamlining and 

technical assistance were options from the written survey, however, so they 

might not have been offered as frequently had PHAs not had the ability to check 

them off. Still, we questioned PHAs during the interviews concerning these 

recommendations and most said they still would make them.

6. Increase PHA exposure to the EPC program.

This recommendation was offered by a few PHAs that felt that at one time HUD 

had focused considerable attention on the EPC program but was no longer doing 

so. They felt that if HUD wants more PHAs to consider the program, it should

educate them again on its merits and why they should seriously consider it.

Other recommendations were received from one or more PHAs, although we have 

summarized the most prevalent. In appendix B, however, we have included summaries 

of our notes from all 20 of the interviews, with only the identities of the PHAs omitted. 

Other PHA suggestions and observations regarding the EPC program can be found 

there.
7-1 

Chapter 7

Analysis of Utility, Financial, and Physical

Conditions Data

Background 

Although the public housing authority survey was the principal means of collecting data 

to address the central questions of this project, the study conducted other analyses 

using HUD administrative data. Specifically, the study gathered Utility Expense Level

(UEL) data as well as HUD internal ratings of PHA financial and physical condition and 

PHA financial data to analyze whether PHAs that used Energy Performance Contracts

performed better in these respects than PHAs that did not.

Analytic Approach 

Analysis of UEL Data

HUD housing units consume various forms of energy, including electricity, natural gas, 

fuel oil, and, in a few cases, coal. We examined HUD’s energy consumption data and 

determined that although electricity data were sufficient for statistical analysis of PHA 

consumption trends, natural gas and other energy-related data were not. We reached 

the latter conclusion because there was less such data, the data sets were incomplete in 

many instances, and some of them were recorded using varying or even inapplicable 

metrics.

1 Hence, our analysis of HUD energy data from administrative sources is limited 

to electricity.

To measure the effectiveness of EPCs with respect to PHA electricity consumption, we 

used UEL data reported by PHAs and maintained by HUD. LMI selected a sample of 

427 PHAs, consisting of 188 EPC PHAs and 239 non-EPC PHAs. Of this sample of 427, 

useful electricity data were available for only 321, including 146 PHAs that had done 

EPCs and 175 that had not. To assess the effectiveness of EPCs, we made two different 

types of comparisons:

• Long-term trends: We compared electricity consumption for all years prior to the 

execution of an EPC with consumption for all years after such execution. Where 

a PHA used multiple EPCs, we compared all years prior to the execution of the 

first EPC with consumption for years thereafter. For the non-EPC PHA group, we 

compared consumption before and after 2010, choosing that year because it was 

roughly the midpoint of our electricity consumption data set. 

• Short-term effect: We compared utility consumption 2 years prior to deploying an 

EPC with consumption 2 years after an EPC. For the non-EPC group, we set the 

base year as 2010, and compared consumption 2 years before 2010 with 2 years 

after 2010.

1 These issues are described more fully in appendix C. 
 

7-2 

Analysis of Long-Term Electricity Consumption Trends

We took the following steps.

Step 1—Data cleansing: We initially processed and sanitized the available UEL data for 

each PHA. For the EPC PHAs, we required an EPC year (the year when an EPC was 

executed) for each PHA record. We observed inconsistencies in the data for when an 

EPC was executed; 76 of the 146 EPC PHAs did not report the year the EPC was 

initiated. Because we did not know when the EPC was begun, we eliminated these 

records and limited the sample to only 70 EPC PHAs.

Step 2—Calculating average consumption across all years: We calculated average utility 

consumption before and after the EPC was executed (or the 2010 base year for nonEPCs) for each individual PHA.

To illustrate, consider FL041. This PHA reported utility consumption for three Asset 

Management Projects (AMPs). For all three, the first EPC year was 2007. The following 

steps were used to calculate the pre-EPC and post-EPC utility consumption averages:

1. We first calculated pre-EPC average consumption in years prior to 2007, 

specifically for 2004, 2005, and 2006. We then calculated average consumption 

for the years after 2007. These data covered the years 2008 to 2016.

2. We summed these consumption totals to derive totals for the PHA and then 

compared the pre-EPC average utility consumption with the post-EPC

consumption. 

Exhibit 7-1 shows the calculations for the pre-EPC and post-EPC average utility 

consumption for FL041. The year 2007 is left blank because that was the year an EPC 

was executed by this PHA. 

Exhibit 7-1. Example Data Set and Calculated Averages for FL041

Electricity Consumption in Thousands of Kilowatt Hours (000 KWh)

Unit 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016

PreEPC PostEPC 

FL041000001 2,371 2,573 2,902 N/A 2,339 2,217 2,403 2,335 2,221 2,248 2,308 2,339 2,448 2,615 2,318

FL041000002 453 478 500 N/A 1,415 1,356 1,462 1,434 1,356 1,439 1,799 1,950 1,651 477 1,540

FL041000003 1,720 1,595 1,644 N/A 1,120 1,009 1,067 944 915 886 933 1,058 1,141 1,653 1,008

Total PHA 

FL041

4,544 4,646 5,046 N/A 4,874 4,582 4,932 4,713 4,492 4,573 5,040 5,347 5,240 4,745 4,866

We then compared average consumption levels for the entire population of EPC-using 

and non-EPC-using PHAs to assess the extent to which consumption dropped after 

EPCs were implemented or after the 2010 base year.

To do so, we used the following formulae:

EPC_pre(all years) = ∑ UUUU (OOOO rraaaa EEEEUU EEPPPPPP bbbbOObbrrbb EEEEUU yybbrrrr)

NNEEEEUU−pprrbb
Analysis of Utility, Financial, and Physical Conditions Data

7-3 

Non-EPC_pre(all years) = ∑ UUUU (bbOO rraaaa nnbbnn−EEEEUU EEPPPPPP bbbbOObbrrbb 2010)

NNNNbbnn−EEEEUUpprrbb

EPC_post(all years) = ∑ UUUU (bbOO rraaaa EEEEUU EEPPPPPP rrOOrrbbrr EEEEUU yybbrrrr)

NNEEEEUU−ppbbPPrr

Non-EPC_post(all years) = ∑ UUUU (bbOO rraaaa nnbbnn−EEEEUU EEPPPPPP rrOOrrbbrr 2010)

NNNNbbnn−EEEEUUppbbPPrr

Where,

NEPC-pre = number of years in the pre-EPC period;

NNon-EPCpre = number of years before 2010;

NEPC-post = number of years in the post-EPC period;

NNon-EPC-post = number of years after 2010 for non-EPC PHAs; and

UC = utility consumption of a PHA.

Step 3—Dealing with outliers: After calculating average electricity consumptions for each 

PHA, we looked at the percentage change from before to after an EPC was executed. 

This percentage change, however, could be strongly affected if there is inconsistent data 

for only a single year. Such inconsistencies occurred through various factors, such as 

deviations from a PHA’s previous consumption trend. Most of the inconsistencies were 

probably due to measurement or recording errors. To avoid results affected by such 

outliers, we sought to eliminate those that were the most extreme. To do so, we defined 

statistical outliers as deviating significantly from year-to-year percentage changes across 

all PHAs. A well-known approach to identify outliers is the 1.5 ÷ IQR rule, where IQR is 

known as the inter-quartile range. We calculated the upper and lower bounds of the 

percentage change data as shown here:

Upper bound = (1.5 ÷ IQR) + Q3

Lower bound = Q1–(1.5 ÷ IQR). 

Where Q1 and Q3 are the first and third quartiles of the percentage change distribution 

across all PHAs, and IQR is the inter-quartile range (Q3–Q1). In other words, we 

eliminated data points that were more than 50 percent above or below the 25th and 75th 

percentiles of year-to-year changes displayed on average by the entirety of the 

population.

Using this method, we detected 12 outliers among EPC PHAs and 19 among non-EPC 

PHAs. After excluding outliers, the final data set consisted of 58 EPC PHAs and 156

non-EPC PHAs.

Step 4—Paired t-test for within-group comparisons: A paired-samples t-test was 

conducted to compare long-term electricity consumption changes in Group 1 PHAs 

before and after an EPC was executed. In this test, we assumed the null hypotheses 

that “There is no difference between average electricity consumption for all years before 

and for all years after an EPC was executed.” We conducted this test at a 95 percent 

confidence level. We also conducted paired-samples t-tests for each of the individual 

size groups: small, medium, and large PHAs. For non-EPC PHAs, as explained 

previously, we conducted paired-sample t-tests to compare consumption during all years 

prior to 2010 and after 2010. The null hypothesis assumes that “There is no difference 

between average electricity consumption for all years before and after 2010.” 
 

7-4 

Findings for EPC PHAs

1. All PHAs (58): There was a significant difference when comparing pre-EPC 

electricity consumption (mean = 5.8 million kWh/yr., standard deviation [SD] = 

10.9 million) with post-EPC consumption (mean = 5.07 million kWh/yr., SD = 9.7 

million); t(57) = 2.764, p = 0.008. This test shows that overall among EPC PHAs, 

there was a statistically significant decrease in annual electricity consumption 

after an EPC was executed relative to before at a 95 percent confidence level.

2. Small PHAs (19): There was a significant difference when comparing pre-EPC

electricity consumption among small PHAs (mean = 0.96 million kWh/yr., SD = 

0.81 million) with post-EPC consumption (mean = 0.86 million kWh/yr., SD = 0.72 

million); t(18) = 2.225, p = 0.039. This test shows that for small EPC PHAs there 

was a statistically significant decrease in annual electricity consumption after an 

EPC was executed relative to before at a 95 percent confidence level.

3. Medium PHAs (19): There was no significant difference when comparing preEPC electricity consumption among medium PHAs before an EPC was executed 

(mean = 2.4 million kWh/yr., SD = 1.7 million) with post-EPC consumption (mean 

= 2.3 million kWh/yr., SD = 1.6 million); t(18) = 1.362, p = 0.190. This test shows 

that for medium-sized EPC PHAs, we did not observe a change in annual 

electricity consumption after an EPC was executed relative to before that was 

statistically significant at a 95 percent confidence level. We did, however,

observe a significant decrease for this size group at a 90 percent confidence 

level.

4. Large PHAs (20): There was a significant difference when comparing pre-EPC 

electricity consumption among large PHAs (mean = 13.6 million kWh/yr., SD = 

15.9 million) with post-EPC consumption (mean = 11.7 million kWh/yr., SD = 14.3 

million); t(19) = 2.64, p = 0.016. This test shows that for large EPC PHAs, there 

was a statistically significant decrease in annual electricity consumption after an 

EPC was executed relative to before at a 95 percent confidence level.

Interpretation of results: The results of these tests indicate that, at a 95 percent or 90

percent confidence level, there was a statistically significant reduction in PHA annual 

electricity consumption when comparing that consumption before and after an EPC was 

executed.

Findings for Non-EPC PHAs

1. All PHAs (156): There was a significant difference when comparing pre-2010

electricity consumption (mean = 3.3 million kWh/yr., SD = 4.8 million) with post2010 consumption (mean= 3.1 kWh/yr., SD = 4.4 million); t(155) = 3.075, p = 

0.002. This test shows that overall among non-EPC PHAs, there was a 

statistically significant decrease in annual electricity consumption after 2010

relative to before at a 95 percent confidence level.

2. Small PHAs (61): There was no significant difference when comparing pre-2010

electricity consumption (mean = 0.8 million kWh/yr., SD = 0.9 million) with post2010 consumption (mean= 0.84 kWh/yr., SD = 0.85 million); t(60) = 1.658, p = 

0.102. This test shows that among small non-EPC PHAs, there was no 

statistically significant change in annual electricity consumption after 2010 

relative to before at a 95 percent confidence level.
Analysis of Utility, Financial, and Physical Conditions Data

7-5 

3. Medium PHAs (60): There was a significant difference when comparing pre-2010

electricity consumption (mean = 2.7 million kWh/yr., SD = 2.2 million) with post2010 consumption (mean= 2.6 kWh/yr., SD = 2.1 million); t(59) = 2.108, p = 

0.039. This test shows that among medium non-EPC PHAs, there was a 

statistically significant decrease in annual electricity consumption after 2010 

relative to before at a 95 percent confidence level.

4. Large PHAs (35): There was a significant difference when comparing pre-2010

electricity consumption (mean = 8.4 million kWh/yr., SD = 7.7 million) with post2010 consumption (mean = 7.8 kWh/yr., SD = 6.8 million); t(34) = 2.454, p = 

0.019. This test shows that among large non-EPC PHAs, there was a statistically 

significant decrease in annual electricity consumption after 2010 relative to 

before at a 95 percent confidence level.

Conclusion: From the tests described previously, we can conclude that there were 

statistically significant long-term decreases in annual electricity consumption after an 

EPC was executed relative to before among small and large PHAs and overall among 

this group. A statistically significant decrease occurred in annual electricity consumption, 

however, among medium and large non-EPC PHAs after 2010 relative to before, and 

overall among such PHAs. Clearly, electricity consumption was decreasing among the 

various PHA populations during the time period under study.

Step 5—Independent samples t-test for between-group comparisons: An independent 

samples t-test was conducted to compare long-term electricity consumption trends 

between the EPC and non-EPC groups. In this test, we assumed the null hypothesis that 

“There is no difference between the average change in electricity consumption before 

and after an EPC was executed for EPC PHAs and the average change in electricity

consumption before and after 2010 for non-EPC PHAs.” We conducted this test at a 95

percent confidence level. We also conducted independent samples t-tests for each of 

the individual size groups: small, medium, and large PHAs.

Findings for Comparison of EPC vs. Non-EPC Electricity Consumption

Comparing the two groups, there was a higher percentage decrease in annual electricity

consumption among EPC than non-EPC PHAs (10.3 mean percent decrease for EPC 

PHAs compared with a 4.4 mean percent decrease for non-EPC PHAs). We used an 

independent samples t-test to test whether this difference was statistically significant and 

found that it was. The outcome suggests that EPCs probably did play a role in reducing 

electricity consumption more rapidly for PHAs that used them during the period and 

particularly for small PHAs. They also may have helped medium- and large-sized EPCusing PHAs reduce their electricity consumption by more than comparably sized nonEPC users, but these differences were not statistically significant. The results are 

summarized in exhibit 7-2.
 

7-6 

Exhibit 7-2. Summary of Changes in Electricity Consumption

Between EPC PHAs and Non-EPC PHAs

Small PHAsa Medium PHAs Large PHAs All PHAsa

Average change in electricity

consumption after an EPC

−11.5% −7.9% −11.5% −10.3% 

Average change in electricity

consumption with no EPC (2010 

base year)

−2.8% −4.9% −6.3% −4.4% 

a Statistically significant difference at the 95-percent level.

Sensitivity test for EPC versus non-EPC comparison: Because the choice of 2010 as the 

before and after dividing year for non-EPC PHAs might have influenced the results 

obtained, we reran the analysis using 2009 or 2011 as the dividing year instead. This 

change made little difference in terms of the change in electricity consumption for the 

non-EPC PHAs (for example, still −6.0 percent for all PHAs using 2009 and −6.4 percent 

using 2011), and in both cases there was a statistically significant difference between 

the rate at which annual electricity consumption declined for EPC PHAs and non-EPC 

PHAs. This result reinforces the conclusion that EPCs did have a meaningful effect on 

longer-term electricity consumption trends for PHAs that used them relative to PHAs that

did not.

Analysis of Short-Term Electricity Consumption Trends

To analyze short-term trends of electricity consumption from using EPCs, we compared 

average annual consumption 2 years prior with 2 years post the year an EPC was

executed. For non-EPC PHAs, we looked at 2 years prior to and 2 years post-2010. We 

repeated steps 1 to 4 as described in the previous section. No statistically significant 

difference was observed between 2 years prior-EPC consumption and 2 years post-EPC 

consumption for all subgroups of PHAs at the 95 percent confidence level. For the nonEPC PHAs, no statistically significant difference in consumption before and after 2010 

was observed across all size groups. We conclude that although electricity consumption 

was dropping on average during the longer period for all PHAs and dropped even more 

for EPC users, the differences are not sufficient during 2-year periods to draw 

conclusions from a statistical perspective.

Several limitations to the statistical findings should be noted.

1. We did not have data for all EPC years. In particular, we did not have UEL data 

for 2008 and 2009 but were able to fill in some of these missing data from past 

year figures in the 2010 data set.2

2. Available UEL data may not have included all energy consumption occurring 

within a PHA. For example, tenant-paid utility data are excluded from UEL data 

although they generally are included in PHA utility allowance calculations.

3. A number of data quality issues were discovered, including gaps and inaccuracy. 

For example, consumption totals sometimes seemed far out of line with the size 

of a PHA or with previous trends for the same PHA. We corrected the data as 

best we could, changing units that seemed unrealistic, checking other years’ 

2 The UEL data sets consist of the current year’s data as well as 3 years of prior data. 
Analysis of Utility, Financial, and Physical Conditions Data

7-7 

reports to see whether a missing or unrealistic number had been included there, 

removing outliers, and so forth.

Analysis of Financial and Physical Condition

REAC Scoring Data 

To measure the impact of EPCs on a PHA’s financial and physical condition, we 

obtained scoring data from the Real Estate Assessment Center (REAC) Online Systems 

(the database is labeled Public Housing Assessment System [PHAS], which includes the 

Physical Assessment Subsystem (PASS), the Financial Management Assessment 

Subsystem (FASS), and the Management Assessment Subsystem [MASS]), and a total 

score that includes these three scores plus a Capital Fund and a Resident score. We did 

not use the Capital Fund score nor the Resident score in our analysis but instead looked 

at each of the other three plus an average of the sum of these three scores over time.

The data covered several years of PASS, FASS, and MASS scores, enabling us to 

compare pre-EPC with post-EPC scores for EPC PHAs, and pre- and post-2010 scores 

for non-EPC users. The idea was that if the use of EPCs materially improved a PHA’s

financial or physical condition relative to non-use of EPCs, this result might show up in 

the respective trends of FASS and PASS scores over time.

Step 1—Data considerations: Several data considerations and processing changes were 

made to compensate for data inconsistencies or missing data:

1. Some PHAs were missing individual subsystem scores; however, overall scores 

and other individual scores were available. For such cases, we inferred 

remaining individual subsystem scores by subtracting available scores from total 

PHAS scores. 

2. The PHAS scoring method was not consistent across the years and was different 

for 2007 and prior and for 2011 and after. Data were not available for the period 

from 2008 to 2010.

­ Total points 2007 and prior: PASS—30 points; FASS—30 points; 

MASS—30 points.

­ Total points 2011 and after: PASS—40 points; FASS—25 points; 

MASS—25 points.

Due to the scoring inconsistencies, PHAS scores were standardized across all 

years using the percentage points they received in each year.

3. Resident score and Capital Fund score are 10 points each but, as stated 

previously, were not used for this analysis.

4. The analysis included 116 EPC PHAs and 2,729 non-EPC PHAs. 

Step 2—Calculating average scores across all years: We calculated average total 

scores before and after an EPC was executed (or the base year 2010 for non-EPCs) for 

each individual PHA. We used the standardized percentage scores for all the analyses 

in this section. We calculated average standardized scores for the pre-EPC and postEPC periods (or pre-base and post-base for non-EPCs) for each PHA using the 

formulae:

EPC_pre_PASS(all years) = ∑ EEPPPPPP (OOOO rraaaa EEEEUU EEPPPPPP bbbbOObbrrbb EEEEUU yybbrrrr)

NNEEEEUU−pprrbb
 

7-8 

Non-EPC_pre_PASS(all years) = ∑ EEPPPPPP(OOOO rraaaa nnbbnn−EEEEUU EEPPPPPP bbbbOObbrrbb 2010)

NNNNbbnn−EEEEUUpprrbb

EPC_post_PASS(all years) = ∑ EEPPPPPP (OOOO rraaaa EEEEUU EEPPPPPP rrOOrrbbrr EEEEUU yybbrrrr)

NNEEEEUU−ppbbPPrr

Non-EPC_post_PASS(all years) = ∑ EEPPPPPP (OOOO rraaaa nnbbnn−EEEEUU EEPPPPPP rrOOrrbbrr 2010)

NNNNbbnn−EEEEUUppbbPPrr

Where,

NEPC-pre = number of all years in the pre-EPC period;

NNon-EPCpre = number of all years before 2010;

NEPC-post = number of years in the post-EPC period;

NNon-EPC-post = number of years after 2010 for non-EPC PHAs; and

PASS = standardized PASS score of a PHA.

The average FASS, MASS, and overall PHA scores for all EPC and non-EPC PHAs 

were calculated in a similar manner, using the same formulae for pre- and postcomparison periods.

Step 3—Detecting outliers: After calculating average scores for each PHA, we looked at 

the percentage change of average total scores before and after an EPC was executed. 

To eliminate inconsistencies from the data, we found statistical outliers from the 

percentage change across all PHAs; that is, data points that lie outside the overall 

pattern in a distribution. As described previously, a well-known approach for finding 

outliers is the 1.5 ÷ IQR rule, where IQR is known as the inter-quartile range. We 

followed the same procedures described in Step 3 (p. 7-3) to detect and eliminate 

outliers from the data set.

Step 4—Paired T-test: A paired-samples t-test was conducted to compare total scores in 

EPC PHAs during all years before and after an EPC was executed. In this test, we 

assumed the null hypothesis that “There is no difference between average total (PASS, 

MASS, FASS) scores for all years before and for all years after an EPC was executed.” 

We conducted this test at a 95 percent confidence level. We also conducted pairedsamples t-tests for each of the individual size groups: small, medium, and large. For 

non-EPC PHAs, we conducted paired-sample t-tests to compare total scores during all 

years prior to 2010 and all years after 2010. The null hypothesis proposes that “There is 

no difference between average total scores for all years before and after 2010.” 

Findings

Exhibit 7-3 shows results that were observed in all the paired-samples t-tests conducted 

across PHAs at the overall level. The PASS scores showed a significant increase in pre 

and post when an EPC was executed. The non-EPC PASS scores, however, also 

showed a significant increase in pre- and post-2010. Management scores (MASS) for 

both groups declined significantly during the respective periods.3 Financial scores 

(FASS) showed a significant increase in the non-EPC group but not in the EPC group. 

Overall, there was no score improvement in the EPC group and a slight increase in the 

non-EPC group. From these data, we are unable to conclude that the use of EPCs 

3 MASS scores are importantly affected by tenant occupancy rates. Other factors include Tenant Accounts Receivable and Accounts

Payable. We did not investigate why MASS scores declined as they were not central to our investigation. 
Analysis of Utility, Financial, and Physical Conditions Data

7-9 

improved a PHA’s physical or financial condition relative to PHAs that did not utilize 

them.

Financial Data: LMI also studied financial data extracted from the HUD Financial Data 

system. This system yielded the following information for each PHA:

• Total revenue

• Total expenses

• Net investment in capital assets

If EPCs have materially improved PHAs’ finances relative to non-use of EPCs, this 

improvement might show up in revenue less expense data, or in net investment data. 

Data were available in two tabulations: before 2013 and after 2013. Data from both 

tabulations were integrated for all years.

Analyses: Prior to the analyses, the data were divided into two groups of PHAs, EPC 

and non-EPC, and further by size: small, medium, and large. For each of the subgroups, 

we calculated:

Available funds = total revenue–total expenses.

We then plotted net available funds across PHAs during the years to observe whether 

trends emerged. Exhibit 7-4 shows available funds during the years since 2002, by PHA 

size class. For all the subgroups, we noticed decreases in available funds in the last few 

years.

Exhibit 7-3. Pre-, Post-, and Non-EPC PASS, MASS, and FASS Scores

EPC PHAs

PASS (n = 112) MASS (n= 114) FASS (n = 111) Overall score (n = 111)

PremeanPostmeanDifference PremeanPostmeanDifference PremeanPostmeanDifference PremeanPostmeanDifference

0.80 0.83 4.010%a 0.88 0.83 −5.803%a 0.84 0.83 −1.402% 0.84 0.83 −0.596%

Non-EPC PHAs

PASS (n = 2,645) MASS (n = 2,626) FASS (n = 2,431) Overall score (n = 2,611)

PremeanPostmeanDifference PremeanPostmeanDifference PremeanPostmeanDifference PremeanPostmeanDifference

0.84 0.87 3.636%a 0.91 0.83 −8.776%a 0.84 0.92 9.084%a 0.86 0.87 0.233%a

a Statistically significant at the 95-percent level.
 

7-10

Exhibit 7-4. Available Funds Over Time to EPC and Non-EPC PHAs

Data source: HUD Financial Data System. 
Analysis of Utility, Financial, and Physical Conditions Data

7-11

Further detailed analyses could be explored to compare available funds pre and post the 

year of EPC execution for EPC PHAs with pre- and post-2010 for the non-EPC PHAs. 

We could use a similar paired-sample t-test to compare means in the pre- with post-time

periods to determine if the effect of an EPC was significant or not. Our reading of PHA 

financial data, however, was that many factors other than EPCs heavily influence both 

available funds and net investment in capital assets, and we had little confidence that 

tests involving EPCs alone would yield meaningful results. If any conclusion could be 

drawn from exhibit 7-4, it is that the financial situation of all PHAs, whether they did 

EPCs or not and regardless of size, deteriorated during the latter years under study. The 

effects on HUD’s budget of federal budget sequestration occurring in 2013 and a few 

years after that probably were important reasons why this decline occurred.

8-1 

Chapter 8

Relationship of the RAD Program to EPCs

Background 

Since 2012, public housing authorities have been able to use an alternative source of 

private investment capital to upgrade units under their jurisdiction. As briefly explained in 

Chapter 2, the Rental Assistance Demonstration program converts PHA-owned

properties to rental assistance under Section 8 of the United States Housing Act of 1937, 

which supplies project-based rental assistance or project-based voucher assistance over 

long-term contracts, typically 20 years.1 These long-term commitments to subsidize 

tenant rents serve as a credit enhancement to attract long-term debt and equity from the 

sale of Low-Income Housing Tax Credits (LIHTCs) in order to attract capital to upgrade 

facilities and reduce backlogs of unfunded capital improvements.2

Although Congress initially limited the RAD program to 60,000 units, in order to keep 

pace with growing demand, Congress has expanded RAD by more than sevenfold, to 

455,000 units, as documented in exhibit 8-1. 

Exhibit 8-1. Numbers of RAD Housing Units Authorized by Congress, by Year

Time Enabling legislation Ceiling on RAD units

Summer 2012 Further Continuing Appropriations Act, 2012 60,000

December 2014 Consolidated Appropriations Act 185,000

December 2016 Consolidated Appropriations Act 225,000

April 2018 FY18 Omnibus Spending Bill 455,000

As of October 2018, PHAs have generated over $7.6 billion towards capital 

improvements across 123,000 converted units. An additional 150,000 units are in the 

process of converting (unpublished PIH data). Further, as of the beginning of 2018, 

around 300,000 units of the 1.1 million publicly owned housing units were in the process 

of converting to RAD (HUD, 2018a). Of these housing units, about 57 percent were 

converting to Project-Based Vouchers and 43 percent to Project-Based Rental 

Assistance (HUD, 2018b). HUD’s expectation is that the number will reach 600,000 by 

2021 should the cap be further relaxed (HUD, 2018b). Given the rate at which Congress 

has expanded the program to date and its obvious desire to attract private capital to 

upgrade the nation’s subsidized housing stock, the 455,000-ceiling likely will not be a 

permanent cap.

1 See for example https://www.hud.gov/RAD/program-details. 

2 Two additional alternatives for accessing private capital for property improvements are not within the scope of this study. The 

Streamlined Voluntary Conversion program offers small PHAs with less than 250 units the ability to convert public housing units to 

assistance with Section 8 tenant-protection vouchers rather than project-based Section 8. Demolition and disposition authority under 

Section 18 also enables PHAs to reposition public housing for financial sustainability and access private capital.
 

8-2 

PHA Options

Because the unit cap was greatly increased in 2018, many more PHAs will be able to 

access public and private capital through the RAD program. Even as it is, more PHAs 

have undertaken RAD conversions than Energy Performance Contracts, however, a 

number have done both or undertook an EPC but later converted the properties to RAD 

and bought out the EPC contract. Exhibit 8-2 provides information on the numbers of 

PHAs that have done one or the other or both. Of those shown as doing both, however, 

60 had completed their EPCs by 2011, whereas the RAD program was only enacted in 

2012. Thus, at most only 33 were doing both at the same time.

Exhibit 8-2. Numbers of PHAs Executing EPCs, Converting to RAD or Doing 

Botha

PHAs that have done 

or are doing EPCs PHAs engaging in RAD PHAs that have done both

260 476 93

a Data sources: Number of PHAs doing EPCs and the year of completion from data supplied by HUD in 2017. The 

database contains 315 separate EPCs but in 55 cases a single PHA had done more than one. RAD data obtained from 

HUD at http://www.radresouce.net/. The number doing both was obtained by comparing the PHA names in the two sets 

of data.

For a PHA deciding whether to convert to RAD, an ongoing EPC project complicates 

matters because of the debt already incurred and because it means having to manage 

two contracts at a given property simultaneously. Indeed, the very presence of ongoing 

EPC-related debt can discourage financiers of RAD projects from getting involved with a

particular PHA in the first place.

The PHA might deal with this issue by refinancing the EPC debt as part of the RAD 

conversion. That simplifies matters for managing projects and erases the debt, but may 

involve costly negotiations over terms of the buyout with the EPC contractor.

Given the present ceiling on the number of RAD units, some PHAs may seek to use 

EPCs for units that are not contemplated for RAD conversion in the near future. This

situation is more likely to be the case for PHAs that on principle wish to retain their 

properties within the public housing program or where RAD rents would be inadequate 

to support the financing for holistic property improvements. The EPC program offers 

such PHAs an alternative private financing option.

Gains in Energy Efficiency from EPCs and RAD 

An EPC contractor is repaid from savings in energy costs, which motivates contractors 

to make energy savings investments that are cost-effective during a set time horizon. 

These investments may not be the most energy-efficient technology available because 

the contractor may view such technology as relatively untried or risky in some other 

manner, and unlikely to achieve an acceptable savings-to-investment ratio (SIR). Thus, 

some potential energy savings may be left on the table. Further, although cash-poor yet 

creditworthy customers are good clients for EPCs, these contracts are focused on 

energy-consuming systems and tend not to achieve synergistic energy savings that 

might be obtained with comprehensively retrofitted or rehabilitated properties.
Relationship of the RAD Program to EPCs

8-3 

Under the RAD program, a PHA obtains investment in a property against both inherent 

land value and a stream of future payments to be derived from rental subsidies provided 

by HUD. Under RAD, PHAs have been able to achieve leverage ratios of as much as 9:1

(HUD, 2016). That is, for every $1 contributed by public housing funds, RAD is 

leveraging up to $9 from sources such as LIHTC, private mortgage debt, accrued land 

equity, grants, and other funding.

The RAD-enabled investments may include no energy efficiency investments, some

energy efficiency investments, or practically all that are available. Thus, a RAD project 

may accomplish more or less energy efficiency than an EPC. A PHA and its consultants 

do not necessarily have the energy expertise of EPC contractors, and therefore might 

not identify as many promising efficiency candidates. Most of the significant capital 

improvement and housing replacement projects under RAD, however, have leveraged 

LIHTC markets. The LIHTC program does not dictate efficiency standards, but most 

states provide preferences or requirements within their Qualified Allocation Plans 

(QAPs) to obtain the credits. A PHA thus often will have to meet minimum energy and 

water efficiency standards established in the QAP if it chooses to make use of LIHTC tax 

incentives in attracting private capital.

HUD has provided both the EPC and RAD programs with incentive mechanisms to 

overcome situations in which residents pay part or all the utilities. Additionally, RAD 

rents with project-paid utilities are set in a way comparable to the frozen rolling base 

provision of the EPC program. Such similarities suggest that the energy savings 

potential of the two programs does not differ greatly. 

Importantly, RAD transformations are often holistic rehabilitations or construction of new 

units. These changes enable such projects to focus on energy consumption over an 

entire building system, in which synergies among energy conservation investments can 

be realized. This achievement implies that RAD investments could achieve more energy 

efficiency than EPCs simply because they can realize available synergies, but the matter

of which program actually achieves more is an empirical question.

Implications 

The previous discussion suggests the following:

• Due to the demand for and success of the RAD program thus far, it is likely that 

this program will continue and expand during the next few years.

• If PHAs near or reach the present 455,000 unit limit, Congress likely will increase 

it further.

• Because they would incur contractor debt and because they would not want to 

manage two separate rehab contracts rather than only one, PHAs contemplating 

whether to use an EPC will be discouraged from doing so if they believe they will 

be able to undertake RAD. This perception implies that the number of EPC 

applications will be reduced from what it was in the past.

• There will, however, be a niche for EPC projects, for PHAs that on principle wish 

to retain their properties within the public housing program or where RAD rents 

would be inadequate to support the financing for holistic property improvements. 
 

8-4 

• EPC contractors will invest in efficiency to the point where expected gains are 

equal to costs but will leave further efficiency gains on the table if they believe 

the incremental costs exceed the incremental savings. RAD projects may contain 

any level of energy efficiency investment depending on the peculiarities of the 

units involved and the financial leveraging choices made by the local PHA.

Evidence 

Qualitative: Stakeholder Views

Conference Participant Feedback

One member of the LMI team attended a RAD conference in Washington, D.C., in 2018. 

This individual discussed the relationship of RAD to EPCs with other conference 

participants such as PHA officials, consultants, and financiers. Below is a summary of 

the feedback received: 

• EPCs can impose complications for RAD financing. Therefore, if a PHA is 

considering RAD, it should not do an EPC at this point.

• Mature EPCs are easier to resolve in RAD. EPCs with remaining large debt, 

however, will be subject to processing and underwriting requirements.

• If a PHA cannot use RAD because of the unit cap or the nature of its housing 

units, an EPC can be a good option for upgrading energy-related equipment. 

• Most EPCs are based on SIRs, which can result in specifying energy system 

components that are above code minimum but not necessarily the most efficient 

equipment available because expected savings from more efficient equipment 

are not expected to cover incremental costs. Were HUD to require PHAs to 

comply with more efficient LIHTC QAP thresholds or with green building 

certification programs accepted by HUD, more energy efficiency could be 

achieved.

• The use of less efficient equipment in an EPC than that specified in QAP 

requirements may necessitate early equipment replacement if the sponsoring 

PHAs later seek to use RAD and LIHTCs to access private capital. Such early 

retirement is in many cases economically wasteful and could be avoided by

requiring PHA compliance with LIHTC efficiency thresholds for future EPCs.

• A recent article, “The Intersection of EPC and RAD: A Roadmap for PHAs,” 

(Bordenave et al., 2016) concluded that converting a project through RAD after it 

has done an EPC is better than undertaking an EPC alone. The study also 

concluded that, in 80 percent of tested scenarios, it is better to undertake a new 

energy project via RAD than through an EPC.

Survey Conclusions

A section of the LMI team’s online survey asked PHAs that had not done EPCs to give 

reasons why they chose not to do so. One of the main answers was that other options, 

such as RAD, were a superior choice for them. The most prevalent answer given to this 

question, however, was that an EPC was not cost-effective from their perspective. In 

such cases, RAD still may have been an alternative because its economic basis is 

different from that of EPCs, but it was not the main reason PHAs did not use an EPC.
Relationship of the RAD Program to EPCs

8-5 

Follow-up Telephone Interviews with Survey Participants

One of the main conclusions drawn from our telephone interviews with 20 PHAs is that 

many, if not most, use RAD as their vehicle of choice to tap into private funds to upgrade 

their housing units. In several cases, the PHAs interviewed were in the process of 

switching from the EPC program to RAD, buying out their EPC contractor in the process, 

and in others, they had compared RAD with EPCs and decided they would go with the 

former.

The main reason given was the broader funding scope of RAD. PHA directors looking to 

upgrade more than their energy equipment find RAD is better suited to their needs than 

EPCs, whose investment objectives are more limited. Some PHAs also indicated that 

RAD is easier to use because paperwork requirements are less. One interviewee stated 

that from their perspective, HUD was no longer emphasizing the EPC program so that 

RAD now implicitly seems to be HUD’s outside funding vehicle of choice.

Quantitative: EPC Approvals

Exhibit 8-3 shows the number of EPCs approved by HUD in each fiscal year between 

2011 and 2018.3

 

Exhibit 8-3. EPC Approvals by Year

Fiscal year EPC approvals 

2011 19

2012 20

2013 26

2014 7 

2015 21

2016 12

2017 14

2018 10

Although one cannot draw strong conclusions from a limited set of years, it seems 

evident that the number of EPC approvals has been decreasing, with the 2016–2018 

numbers only a little more than one-half those of 2011–2013. Although there could be 

many reasons for such a decline, it is consistent with the proposition that RAD has 

impacted the level of EPC activity. 

3 For years 2011 through 2016, the Energy Center provided EPC project completions by year. For those years, we assumed the 

project applications were approved in the year before the projects were completed. For 2017 and 2018, the Energy Center provided 

us actual numbers of HUD approvals. 

9-1 

Chapter 9

Conclusions and Recommendations

Conclusions

This study has focused on two basic questions asked by HUD.

1. Are the utility, financial, and physical conditions of public housing authorities that 

have executed an Energy Performance Contract better relative to PHAs that 

have not executed an EPC?

2. How much of an impact do the implementation factors of existing EPCs have on 

the program’s adoption rate?

To address these questions, the LMI team used four different methods of acquiring data. 

These approaches included an online survey, follow-up telephone interviews, 

examination of administrative data, and assessment of the effect of the Rental 

Assistance Demonstration on the EPC program. From the results obtained, the LMI 

team reached the following conclusions:

1. Statistical tests performed using HUD Utility Expense Level data showed that 

PHAs that have implemented EPC projects have been able to cut their electricity 

consumption by more than twice as much as PHAs that have not (10.3 percent 

as compared with 4.4 percent). This conclusion is backed at a 95-percent level of 

statistical confidence.

2. Tests using UEL data for EPC users showed electricity consumption declined at 

a significantly greater rate in the years after an EPC was performed than in the 

years immediately before. This result held at a 95 percent level of statistical 

confidence for all such PHAs as well as for small and large ones, but only at a 

90-percent level of confidence for medium-sized PHAs.

3. Analysis of PHA survey data indicates those that performed EPCs reduced their 

use of natural gas and water more than non-EPC users, at a 95-percent level of 

statistical confidence. Survey data indicated that EPC-using PHAs reduced their 

use of electricity and fuel oil by more than non-EPC PHAs as well, but those 

results were not statistically significant.

4. Survey data suggest that, during the past several years, those PHAs that have 

used EPCs have improved their financial condition by somewhat more than those 

that have not. Surveyed EPC-using PHAs also indicated that the financial 

condition of units included within the EPC improved more than their other units. 

These findings were not conclusive at a 95-percent level of statistical confidence, 

however. They also could not be confirmed through examination of administrative 

financial data.

5. PHAs that executed EPCs reported greater improvements in the physical 

conditions of their properties than PHAs that did not. The difference in reported 

improvements between the two samples was significant at a 95-percent level of 

statistical confidence. 
9-2 

6. Approximately 60 percent of PHAs that had done at least one EPC indicated they 

were favorably disposed toward doing another. Because at least some of the 

other 40 percent that had done at least one could already have accomplished all 

the energy investments they considered cost-effective, it is likely that more than 

60 percent found the EPC program useful

7. According to survey evidence, about two-thirds of PHAs that had done an EPC at 

some units but not others indicated that the EPC resulted in lower utility 

consumption and better physical and financial conditions than occurred in their 

non-EPC units. These responses furnish a strong rationale for why most of the 

EPC-using respondents expressed a willingness to do another.

8. The RAD program is being used by a large and growing number of PHAs. 

According to interview data, the RAD program offers a wider application of 

monies to housing units’ physical needs and is easier to negotiate than an EPC. 

The annual number of EPC approvals has declined over time as more PHAs

have turned to RAD.

9. Few PHAs are likely to undertake both RAD and an EPC. Indeed, PHAs that 

have undertaken RAD often have bought out their EPC contracts.

10. Of those PHAs that have never undertaken an EPC, the most important reason 

given was that such a project would not be cost-effective from their perspective. 

Other leading reasons were that the process is too complicated or that RAD is a 

better option.

11. Of those PHAs that started but did not complete an EPC, the main reason given 

was that they did not have sufficient staff to see the project through. Another 

reason was that HUD’s EPC process is too cumbersome.

12. Interviewed PHAs indicated that HUD’s EPC program should be among the 

options available. Interviewees suggested, however, that because many PHAs,

particularly smaller ones, do not fully understand the program, do not have the 

staff to undertake one, or do not know how to deal effectively with contractors, 

more technical support would be useful.

13. Survey respondents and interviewees identified several means by which HUD 

could promote the use of EPCs. These methods include continued 

encouragement to PHAs to undertake the program, direct subsidies, a 

streamlined application process, and greater technical assistance.

Recommendations

The LMI team offers the following recommendations:

1. Because there is strong statistical evidence that EPCs have helped PHAs to 

improve their units’ energy efficiency, the program appears to be accomplishing 

one of its principal purposes. If gains in energy efficiency remain an important 

HUD goal, the EPC program should be maintained if not strengthened.

2. The program offers PHAs a means to tap into private capital markets. The RAD 

program also does so, but not all PHAs are interested in RAD or able to take 

advantage of it. For these PHAs, it is important to maintain the EPC program as 

an option.

3. According to some interviewees, HUD is no longer emphasizing its EPC program 

to PHAs to the extent it once did, and a number of PHAs are skeptical of 
Conclusions and Recommendations

9-3 

undertaking it. These interviewees assert that increased communication 

regarding the program, particularly with respect to lessons learned and how to 

deal with contractors, would lead more PHAs to consider the program. In 

addition, several interviewees believe that more financial and technical support, 

particularly directed toward smaller PHAs that seek to self-manage EPCs, likely 

would increase participation.

4. We did not compare utility savings achieved by RAD-using PHAs with those 

using only EPCs. A possibility exists that RAD-related savings might be greater 

because RAD transformations are often holistic rehabilitation or construction of 

new units in which synergies among energy conservation investments can be 

realized. The comparison should be made to determine whether the RAD 

approach achieves as much if not more energy efficiency than the EPC 

approach.

5. We did not examine whether who paid utility bills between a PHA and the tenants 

made a difference in the PHA choosing to do an EPC. HUD’s Resident-Paid 

Utilities Incentive program (RPU) is intended to deal with this issue, but the 

extent to which PHAs have taken advantage of the incentive is unclear. Even 

with the incentive, however, PHA payment of utilities may provide greater 

inducement to undertake an EPC than when tenants pay the bills for individually 

metered utilities. This issue should be examined by HUD to determine whether 

PHAs with partial or full tenant utility payments are less inclined to undertake 

EPCs.

A-1 

Appendix A 

Responses to Survey Questions

This appendix presents the results of the online survey. For each question, the relative 

frequency of responses is shown in graphical form. Where applicable, responses for 

Group 1 respondents are shown in blue, those for Group 2 in yellow, and those for 

Group 3 in red. Most questions in the survey were directed at one or another of these 

three groups although a few were directed at all of them. The color coding is meant to 

enable the reader to see at a glance which of the three responded to that particular 

question. In each case, the number of the question and the number responding is 

shown. Certain questions are grouped together and listed more than once because their 

responses were used to compare Groups 1 and 3 with one another.
A-2 
Responses to Survey Questions

A-3 
A-4 
Responses to Survey Questions

A-5 
A-6 
Responses to Survey Questions

A-7 
A-8 
Responses to Survey Questions

A-9 
A-10
Responses to Survey Questions

A-11 
A-12
Responses to Survey Questions

A-13 
A-14
Responses to Survey Questions

A-15 
A-16
Responses to Survey Questions

A-17 
A-18
Responses to Survey Questions

A-19 
A-20
Responses to Survey Questions

A-21 
A-22
B-1 

Appendix B 

Summary Notes from Survey Follow-up

Telephone Interviews

Group 1: Have Done an EPC 

Small PHAs

Small 1

• Public housing authority conducted two Energy Performance Contracts that 

resulted in enough savings to cover their costs.

­ The first EPC was self-managed and the second was done in conjunction 

with several other PHAs through an energy service contractor.

­ The EPCs have resulted in the improved financial condition of the PHA.

­ PHA does not need more EPCs. The only remaining improvements would 

have a very low payback.

• PHA has also done other energy conservation work using programs through its 

utility company.

­ Completed weatherization of units at no direct cost to the PHA and was able 

to remove that from the EPC project.

• PHA has converted to the Rental Assistance Demonstration, which is viewed as 

a financial and administrative change that does not affect the day-to-day 

operations.

­ The first EPC was paid off before converting to RAD, and the second was 

bought out through the RAD process. Buying out the EPC went smoothly and 

did not present any issues.

• Recommendations for improving the EPC program:

­ Provide training to the tenants. Tenant behavior has a big impact on how 

successful the EPC can be.

­ Education or assistance for the PHA with the procurement process.

 When procuring an energy service contractor (ESCo), PHAs need to 

know how to get the best value for the work being conducted by the 

contractor.

 ESCos may have preferred bidders and there could be opportunities to 

save cost by requiring the ESCo to expand the bidder pool.

 Opportunities can be found through thorough questioning of the ESCo 

bidding process and the PHA being well informed. The EPC will be paid 

off faster if expenditures are limited from the beginning.
B-2 

 Those administering the EPC at the local level need to question and be 

aware of exactly what the ESCo is offering and realize they are also trying 

to make a profit.

­ Account for distribution cost in savings calculations because utility distribution 

costs can increase faster than commodity costs.

­ PHAs need to be very dedicated to make sure everything in the EPC process 

is done as efficiently as possible.

Small 2

• The EPC process was started in 2005 and was completed through an ESCo. 

­ Savings were enough to cover the cost of debt with some extra left over.

­ Had some issues after the EPC installation with the expected level of 

continued tenant education.

 Education of tenants is important because they drive the level of 

consumption.

 Continuous tenant education was included in the EPC contract but did not 

meet expectations.

• Believe that the EPC program has been more or less taken over by the RAD 

program.

­ Unsure if the two could be used together. The financing and details would 

have to be discussed very early in the project to figure out if they would work 

together.

• Recommendations for improving the EPC program:

­ Tenant education because tenants need to be aware of what the EPC is 

doing for them, especially if they pay utilities.

 Education on how to conserve energy and be more efficient.

 PHA used an education awareness team that employed young people 

from the community to help educate tenants.

 PHA included an Energy Fair in the EPC contract to put on an 

educational event for the tenants after the EPC installation to help 

familiarize tenants with new products.

Small 3

• The EPC was conducted in 2000 as a 12-year contract.

­ Believe EPC went smoothly and did not feel that being a small PHA 

presented any additional challenges with completing the EPC.

­ Energy and water savings exceeded expectations.

­ Had issues with the Energy Information and Performance Center system 

(online platform) when that was implemented.

­ View having an EPC as a potential hindrance when updating and transferring 

the portfolio to RAD.
Summary Notes from Survey Follow-up Telephone Interviews

B-3 

• PHA did not extend the existing EPC and would not consider another EPC 

because they have converted to RAD.

­ Able to do more wholesale improvements under the RAD program.

­ Have done modernization under RAD which has included energy savings.

­ Believe that PHAs can choose RAD or an EPC but not both.

• Recommendations for improving the EPC program:

­ The interviewee did not directly work the process of obtaining an EPC but 

believes it could be more streamlined.

­ More technical expertise could be provided.

 Field office could not provide the assistance needed when the EPC was 

executed.

­ The EPC and RAD programs need to be more coordinated. The programs 

come out of different offices in HUD but they are not communicating among 

themselves.

Medium PHAs

Medium 1

• The PHA has an EPC but is currently converting to RAD and working to buy out 

the EPC.

­ Unlikely to do another EPC because of the RAD conversion.

­ Had considered another EPC in the past but struggled to find enough savings 

to make it worthwhile because the PHA is newer than other PHAs (1970s).

­ Under RAD, the PHA will be able to do more deep energy efficiency projects.

 RAD will fast track about 10 years of deep renovations, including 

cosmetic and exterior in addition to energy savings upgrades; these 

renovations would not be possible under an EPC.

• Recommendations for improving the EPC program:

­ Many PHAs do not want to do an EPC because of lack of understanding of 

the program.

­ HUD needs to provide a more holistic and organic approach to explain the 

importance of EPCs and dispel the myth that EPCs are a scam.

 EPCs are important because RAD is not working for a lot of authorities 

and there are a lot of opportunities for savings through an EPC.

 Suggests using social media or podcasts to broadcast success stories of 

the EPC program.

 PHAs fear that tenants will complain about disruptions during the 

implementation of an EPC and that will lead to disciplinary action from 

HUD.

­ Provide incentives for tenants to recompense above-average disruptions.
B-4 

Medium 2

• The EPC was completed in 2012 but the director who executed it is no longer at 

PHA.

­ Feel like they are still paying for improvements that they are not getting much 

benefit from anymore.

­ Tenants get most of the benefits of the savings from the EPC.

­ Would consider an EPC again if it would improve housing for the tenants.

• Considered RAD but could not get funding through the state to convert.

• Recommendations for improving the EPC program:

­ PHAs need to be sure that when implementing an EPC they are getting the 

most value for the contract and a guarantee that improvements made will 

have a longer life than the length of the contract.

Large PHAs

Large 1

• Completed one EPC and currently undergoing a second EPC which is larger in 

scale and value.

­ The first EPC was as conservative as possible to reduce risk.

­ Water cost savings greatly exceeded expectations because of large 

increases in water rates, enabling the PHA to expand the EPC.

­ All utilities are owner paid so the PHA received all the benefit of the savings.

­ The EPC is self-managed.

• RAD was considered and led to a pause in the second EPC project.

­ EPC expansion was limited to only a couple of projects before they were 

converted to RAD.

­ For others, private equity that would be invested was greater under the EPC 

than under the RAD program.

­ RAD is complicated and how it integrates with EPCs is a further complication. 

Also, it can take a long time to achieve RAD conversion (at least 5 years).

­ EPCs were structured so that they could be paid out under RAD if that 

decision is made in the future.

• Recommendations for improving the EPC program:

­ Improve understanding of how the savings are passed through to the PHA 

from HUD.

 Many PHAs think that with the utility savings, HUD will provide less 

money for utilities, meaning the PHA does not get the benefit of the 

savings while still having to pay off the EPC.

­ PHAs need to understand how to capture operating and maintenance (O&M) 

savings through an EPC—O&M savings are not always included in the EPC. 
Summary Notes from Survey Follow-up Telephone Interviews

B-5 

PHAs also think HUD will take away unneeded O&M funds resulting from an 

EPC.

Large 2

• The EPC was completed by 2008 and covered most of the units in the PHA.

­ Electricity is tenant-paid whereas water is owner-paid, with most of the EPC 

savings coming from water and the PHA getting most of the benefit from the 

savings.

• The PHA started a second EPC but did not finish it because of converting to 

RAD.

­ Many of the improvements needed were not energy-related.

­ Saw RAD as a stable funding option moving forward (with unknown future of 

capital funding).

­ Paid off the EPC contractor for some of the RAD properties while holding off 

on converting others to RAD because energy savings would be the main 

component.

­ If the PHA was not doing RAD, it would probably do another EPC but the 

upgrades needed now do not have energy savings associated with them.

• Recommendations for improving the EPC program:

­ Address the stigma that an EPC is taking on debt. Leadership at many PHAs 

will refuse to look at the EPC program because it does not want to take on 

debt.

­ The EPC program is no longer on the forefront of communications. It seems 

as if the program has come and gone.

­ Guide PHAs on how to weigh the options between RAD and an EPC.

 RAD is not a good fit for many authorities and HUD needs to 

communicate to PHAs that if they are not doing RAD, they should 

consider an EPC.

­ Streamline the process for small PHAs—have a simpler process if an EPC 

applies to less than a certain number of units.

­ Train the field office staff so that they understand what an EPC is when PHAs 

ask questions.

 In some cases, field offices are unable to promote the EPC program 

because of lack of understanding.

Large 3

• Completed an EPC that covers about 99 percent of the housing portfolio.

­ Did not have any real issues with the project set up or implementation; 

partnered with other PHAs in a self-managed EPC.

­ Getting residents and maintenance staff behind the project took some effort.

 Tenants complained about low-flow fixtures.
B-6 

 Some of the equipment was complex and required training for 

maintenance staff.

­ Used the Energy Center for the second phase of the EPC and thought it 

provided a good resource and was very responsive.

­ Made further capital improvements from the savings of the EPC and freeing 

up funds to do other improvements that do not have as much utility saving 

payback.

• The PHA considered RAD but decided it was not a good fit; may reconsider in 

the next year.

­ EPC and RAD could be done together but would require more research into 

the programs.

• Recommendations for improving the EPC program:

­ HUD needs to provide more clear guidance between the RAD and EPC 

program and the potential benefit of each; programs need to be better 

integrated.

 Pressuring PHAs to use RAD or an EPC can put them in analysis 

paralysis and they end up not taking advantage of either program.

 The RAD program causes PHAs to have second thoughts about the EPC 

program.

­ Have the subsidy from the EPC go directly to the bank or lender instead of 

through the PHA; this approach would help get more lenders on board with 

the EPC program.

­ Engage residents and maintenance staff early in the EPC planning process.

Group 2: Started but Did Not Complete an EPC

Small PHAs

Small 1

• Did not finish the EPC; decided to go with RAD. All changes that would have 

been included in the EPC were incorporated into the RAD project.

­ The EPC would not have been able to address all the changes needed, only 

small changes.

­ RAD provided more economic benefits.

­ Utilities are tenant paid and tenants love the renovations and utility savings.

• Could not provide any recommendations for improving the EPC program.

Small 2

• Did not finish the EPC because it could not address the full needs of the project 

and was able to convert to RAD to accomplish the tasks planned for in the EPC 

along with other infrastructure needs that would not have been covered by the 

EPC.
Summary Notes from Survey Follow-up Telephone Interviews

B-7 

­ The EPC process was a lot of paperwork and more of a repair than 

addressing the full needs of the buildings.

­ Did not see the value of completing an EPC for a small PHA.

• Strong preference for the RAD program.

­ Much broader and able to bring housing to a higher standard immediately 

(eliminated $80 million worth of deferred maintenance).

­ Addressed all needs, not only energy.

• Recommendations for improving the EPC program:

­ HUD would need to provide a large subsidy for completing an EPC. The RAD 

program is a much better option.

Small 3

• The PHA started the EPC process in 2009/2010 working with an ESCo and does 

not know why the EPC was not completed.

• The PHA was slightly likely to try an EPC again but the RAD program is definitely 

something it is going to pursue (starting to look into it).

­ EPCs are a challenge for a small PHA with little staff and funding.

• Recommendations for improving the EPC program:

­ Need more direct support from HUD, to the point of HUD coming in and 

executing the EPC. As a new executive director, little was known about the 

EPC program and the program was not understood.

Medium PHAs—Medium 1

• Did not finish the EPC because the economics were not there because utilities 

are tenant-paid. The PHA would not have benefited from energy savings. If the 

utilities were owner-paid, it would have made a huge difference.

• The PHA is currently looking into RAD and would plan to achieve energy savings 

through that program.

• Could not provide any recommendations for improving the EPC program.

Large PHAs—Large 1

• Did not complete the EPC because two of the larger projects included in the EPC 

were being identified for redevelopment in the transformation plan.

­ Considered using RAD to execute the transformation plan but determined 

that the program was not right for these projects.

­ Did not have anything against the EPC program but it did not make sense 

considering RAD was being pursued for the remaining projects included in 

the EPC.
B-8 

­ The EPC would have only covered water conservation, now either project is 

going to be redeveloped or RAD improvements have covered both water and 

energy improvements.

­ Felt that the PHA was bearing most of the risk and the ESCo was not.

• Recommendations for improving the EPC program:

­ Provide more training and education about EPCs to the field offices. 

Received a lot of information about EPCs from the ESCo although the field 

office did not have a good sense of how an EPC worked.

­ Provide guidance on how to manage a third party (ESCo).

 What is a good deal, what should be expected, what is standard, and so 

on. 

Group 3: Never Did an EPC 

Small PHAs

Small 1

• The PHA looked into an EPC but decided against it because the PHA had 

enough available cash from capital funding to implement needed energy-saving 

measures.

­ Installed water meters and low-flow showerheads/aerators, replaced toilets 

as needed (estimate 80–90 percent low flow now), capped outside spigots, 

changed to LED lighting as needed, high-efficiency windows, and so on, and 

units are well insulated.

­ Have 198 units and do not have any additional energy conservation needs 

that would warrant an EPC.

 Lack of return on investment.

 The administrative burden would be too high for a small PHA; more 

effective to implement energy conservation measures on their own than 

to complete all the paperwork required for an EPC.

• PHA converted to RAD last year, closing in November 2017.

­ Capital fund alone was enough to do what has been done for energy but not 

enough to do all other desired improvements at the PHA.

­ Allowed them to get out from under unfunded mandates of public housing.

­ After the first year, expect to already have excess funds for other 

improvements.

• Recommendations for improving the EPC program:

­ Allow for direct payments so that PHA can act as its own contractor and use 

direct funding to do all the units at once. Would prefer hiring temporary 

employees or using housing staff to carry out an EPC because of bad 

experiences working with contractors (for example, using less than spec 

equipment).

­ The administrative burden would be too high for a small PHA.
Summary Notes from Survey Follow-up Telephone Interviews

B-9 

• Looking into the EPC program helped the PHA realize the energy improvements 

they could make and at least opened eyes to what they could do to save energy 

for them and their tenants.

Small 2

• Did not know about the EPC program or if anyone had looked into it in the past.

­ Have completed other energy investments (new heat pumps, insulation, 

windows, and so on) but have not done any analysis of resulting savings.

­ Units are in good condition, have been managed well, and money spent 

wisely.

• Looked into the RAD program but prefer to keep units as public housing.

• Does not have any problems with the programs coming out of HUD.

Medium PHAs

Medium 1

• PHA looked into the EPC program about 10 years ago but decided against it 

because of adequate funding through normal capital funds.

­ Energy savings investments incorporated into normal capital projects.

­ Not opposed to the EPC program but have not looked into it more recently.

• PHA had attempted the RAD program but the deal fell through due to 

contaminated soil on the property.

­ Not considering RAD again because capital funding has been enough (even 

increasing).

­ Very disappointed by the amount of time spent on RAD that ultimately was 

not successful. 

• Recommendations for improving the EPC program:

­ Simplify the process and provide more assistance. The PHA’s administrative 

staff is barebones and do not have time to review the process and consider 

whether it is worth the investment.

­ The Energy Center should do as much upfront work for the PHA as possible 

so the PHA does not have to take on the full administrative burden; anything 

to reduce the administrative burden for small and medium PHAs would help.

Medium 2

• Unsure if the PHA has looked into the EPC program in the past because there 

has been complete management level turnover within the last year.

­ The only experience with EPCs was from talking to other PHAs that were 

buying out their EPCs prior to converting to RAD; does not know much more 

about the EPC program.

­ PHA is looking to convert to RAD and redevelop properties within the next 

5 years so would not consider doing an EPC.
B-10

­ Experience is that people do not understand EPCs if they do not already 

have institutional knowledge of the program at the PHA.

• Recommendations for improving the EPC program:

­ Provide more communication, education, and training about the EPC 

program. PHAs are always looking for ways to improve their housing and for 

new sources of funding.

Medium 3

• The PHA has not looked into an EPC because it does not have the staff to review 

the process; EPCs have been on the radar for a long time but the PHA does not 

have the resources to research.

­ Has completed other energy projects with the local energy company (grant 

type program) to improve insulation.

­ Incorporates energy efficiency into any project that is conducted through a 

normal budget.

• Working toward RAD for about 25 percent of units—has been an attractive option 

because units are very old and have high maintenance costs.

• Recommendations for improving the EPC program:

­ Providing an operating subsidy would help get an EPC up and running. 

Starting an EPC is time-consuming and the PHA would need to take 

someone away from their regular responsibilities.

Large PHAs

Large 1

• Looked into the EPC program and is currently interested in figuring out how to 

incorporate an EPC as a component of RAD work being done.

­ Anticipates the RAD program to take a long time but could do an EPC in 

projects scheduled further out.

­ Going to try to combine RAD tax credit with an EPC for a building that only 

needs renovations (not complete teardown and rebuild).

• Recommendations for improving the EPC program:

­ Improve communication and exposure of the EPC program; many PHAs have 

heard about the program but do not understand the “how.” 

­ Develop and implement training and tools/provide technical assistance to 

help PHAs take on a project as sophisticated as an EPC.

­ Provide help to smaller PHAs that do not have the resources to do anything 

other than manage their properties (that is, like looking into programs such as 

EPC and RAD).
Summary Notes from Survey Follow-up Telephone Interviews

B-11

Large 2 

• Considered an EPC in 2008 and completed training. Evaluated capacity of staff 

and determined that the PHA did not have the capacity to take on the project. 

Talked to other PHAs and found that— 

­ some were disappointed by the monetary benefit of the program, and

­ in-house maintenance staff had a hard time understanding new equipment 

installed during an EPC.

• The PHA has completed energy conservation projects through normal operating 

funds or other grants/subsidy money.

­ Includes energy improvements when doing any project.

­ Used grant funding to do bathroom improvements.

­ Have to consider costs and potential savings when incorporating energy 

efficiency into projects (for example, in replacing water heaters, the PHA went 

with a less efficient model because savings from more efficiency would have 

never generated the payback needed to justify the additional cost).

• Working to convert to RAD—have the application submitted.

­ Currently only for part of the PHA, but if it goes well, will probably go RAD for 

all units.

­ It is hard for a PHA to manage more than one investment program at once 

(that is, RAD and EPC).

• Recommendations for improving the EPC program:

­ Provide technical expertise—it is hard finding and hiring qualified personnel 

or contractors to fulfill needs; PHAs could use direct help with building 

system-level technical expertise to be able to oversee an ESCo. 

C-1 

Appendix C 

UEL Data Analysis

To supplement our survey analysis, Utility Expense Level data were reviewed to 

determine if there is evidence of decreased utility consumption when comparing before 

and after an Energy Performance Contract EPC implementation at a public housing 

authority. Utility consumption was also evaluated for non-EPC PHAs, comparing 

consumption before and after an established base year. To undertake this analysis, a 

good deal of data cleansing was necessary. This appendix explains the data cleaning 

and analysis process as well as data limitations and potential ways to include more of it.

Source Data and Cleaning 

UEL data reports were available for funding years 2008, 2011, 2012, 2013, 2014, 2015, 

2016, and 2017. Each UEL data report contains actual annual consumption data for 

various utilities (energy, water, etc.) at the Asset Management Project level. 

Consumption data reflects the year prior (months 7/1/20XX through 6/30/20XX) to the 

funding year (that is, the funding year 2008 UEL data contains consumption data for 

7/1/2006 through 6/20/2007) as well as the 3 years prior.

UEL data review was limited to data for PHAs that were included in the EPC survey 

sample. Sample PHA UEL records for each available year were combined into one file 

and data was scrubbed for inclusion in the analysis. Data cleaning steps included— 

• review of utility type and unit of consumption. Where possible, units or utility type 

were corrected.

• standardization of unit and utility type nomenclature.

• exclusion of flat-rate consumption.

• exclusion of duplicates.

• exclusion of AMPs without data records for each available year of data. Because 

data was rolled up to the PHA level, AMPs with only some years of data had to 

be excluded because they would have skewed utility consumption (that is, 

showing an increase/decrease because of their addition or removal in a given 

year and not because of actual changes in consumption).

Based on our assessment of the UEL data, we determined that only electricity data 

could be included in the statistical analysis. Exhibit C-1 provides an explanation.
C-2 

Exhibit C-1. Data Modifications Made

Utility type in UEL data Data review explanation

Electricity Determined useable records via data cleaning steps.

Gas Data quality issues noted in addition to data cleaning rules:

• Inconsistent units (100 cubic feet; 1,000 cubic feet; cubic feet; flat 

rate, gallons, kWh, mbtu, other/unspecified, decatherm, lbs, 

therms) for AMPs year over year. 

• Unable to determine the correct reporting unit. Excluded AMP 

data if unit changed year over year.

• Data that had all 8 years of records recorded in the same unit of 

measure but with extremely wide ranges of year-over-year 

consumption. For example, if consumption increased by an order 

of magnitude, we took it to mean that the reporting unit in at least 

one of the years likely was incorrect.

• Unable to determine whether records were accurately reporting 

natural gas in appropriate units without additional information 

about the building(s). (That is, gross square feet to compare with

the Department of Energy’s Commercial Building Energy 

Consumption Survey).

Only 36 EPC PHAs had sufficient data available for analysis after 

excluding outliers and PHAs without EPC year.

Fuel Oil Inconsistent units (flat rate, gallons, Therms, other/unspecified) and 

limited PHAs with fuel oil records (50 PHAs total before excluding 

based on data cleaning steps).

Propane Only 15 PHAs total with propane records, many AMPs without 

records for each year.

Lighting listed in UEL data as 

Security Lights, Other (Common 

Area Lighting), Other (Street 

Lights), Other (Site Lighting), Other

(Street Lighting), Other (Outdoor 

Lighting), Other (Outdoor Lights) 

Majority of data reported as a flat rate. A few records that were not 

reported as flat rate included a very limited number of AMPs (less 

than 15 AMPs) that had sporadic annual records (for example, 1–2 

years of data for each AMP).

Diesel listed as Other (Diesel) and 

Other (Diesel Fuel)

3 records: 1 year for one AMP and 2 years for another AMP.

Steam 87 records, unable to determine unit (other, 1,000 lbs, lbs, 100 cubic 

feet, MWH, kW, flat rate, BTU)

Coal Records for only one AMP.

Other (Solar) Unknown utility type. Few (23) records with varying unit—kWh, 

Therms, flat rate, cubic feet.

Other (Stove Gas) Records for only one PHA.

Water Records

Sewerage and Water (water only, if 

sewer reported separately)

Unable to compare water records because of varied reporting at the 

AMP level as water/sewer combined, water only, or sewer only year 

to year. Unable to determine if consumption reflected water use, 

water use plus sewer, or just sewer in UEL records due to data 

quality.

Water and Sewer (if combined) Same as above.

Water (if reported separately) Same as above. 

Sewer (if reported separately) Same as above. 

Sewer Same as above
UEL Data Analysis

C-3 

Exhibit C-1. Data Modifications Made

Utility type in UEL data Data review explanation

Other/Fees listed as Other (Waste 

Water Treatment), Other (Storm 

Drainage Fee), Other (Storm 

Drainage), Other (Stormwater), 

Other (Stormwater Fees), Other 

(Sprinklers), Other (Stormwater 

Utility)

Flat rate utilities.

Other/Unknown Utility Type Records

0 Unknown utility type.

Other (specify) Unknown utility type.

Other Unknown utility type.

Select a Utility Type Unknown utility type.

Other (Fire Protection), Other (Fire 

Service), Other (Fire Line)

Flat rate utility.

Other (Boiler treatment) Flat rate utility.

Other (FLAT RATE) Unknown utility type/flat rate utility.

44 Unknown utility type. 

Other (Landfill User Fee ) Flat rate utility.

Electricity Data Analysis

After excluding unusable records from the analysis, electricity UEL records were 

expanded to include additional years based on the three previous years of available data 

for each funding year record. Consumption for years 2004, 2005, 2006, 2008, and 2009 

was able to be added for AMPs where the base years were listed as non-frozen.

Electricity consumption at the PHA level was analyzed for two groups: PHAs that have 

performed a PHA (survey sample “Group 1”) and PHAs that have not completed an EPC 

(survey sample “Group 2” and “Group 3”). Electricity consumption data were summed to 

the PHA level and average consumption calculated before and after the EPC execution 

or a fixed-base year for non-EPC PHAs. For the non-EPC PHA analysis, 2010 was 

chosen as the fixed base year. Where a PHA had completed more than one EPC, only 

the first execution year was used in the analysis.

The analysis was performed as follows:

• Electricity consumption for all years before and after the EPC/base year was 

averaged at the AMP level.

• Average AMP level consumption was summed to the PHA level.

• The percent change in consumption was calculated for each PHA.

• Outliers were determined using interquartile ranges of the percent change.

• Final results show the percent change in electricity consumption after an 

EPC/base year for all PHAs and broken down by size (small, medium, large) by 

taking the average of the percent change in consumption at each PHA.
C-4 

• Levels of statistical significance were calculated for differences in reductions in 

electricity consumption between EPC and non-EPC PHAs. These levels were 

reported in the text in Chapter 7, exhibit 7-2, and are also shown in exhibit C-1.

Exhibit C-2. Results of Statistical Analysis
UEL Data Analysis

C-5 

Data Limitations

Many limitations and uncertainties exist with the UEL data analysis. For this reason, UEL 

data was supplementary to the survey data collected from PHAs. Known limitations 

include: 

• unknown sizes (gross square footage) of building(s)/AMP year over year; 

• differing data availability for before/after an EPC or variable base year (for 

example, EPCs conducted in 2005 will have limited data before the EPC. 

Likewise, EPCs conducted in 2004 or prior will have no data before the EPC and 

could not be included in the analysis); 

• limited utility types analyzed based on data cleaning rules; 

• missing EPC implementation year in EPC data, limiting the number of EPC PHAs 

that could be included; 

• unknown if UEL data accounts for all utility consumption (that is, some utilities 

may be fully or partially tenant paid); and

• unknown which or how many AMPs affected by EPC implementation.

Further Analysis 

Because the main source of data for this project was intended to be a broad survey of 

PHAs, we were limited in how much UEL data we were able to access, cleanse, and 

analyze. Further efforts to work with UEL data might yield a larger sample and a more 

robust analysis. These efforts could include: 

• refine data cleaning rules to be able to include more records and utility types 

(natural gas, fuel oil, water, and so on) in analysis; 

• continue to fill data gaps further with data about 3 previous years provided in 

each UEL report; 

• couple analysis with other HUD utility data that may be available (for example,

utility allowance calculations from form HUD-52667 and PHA’s per unit month 

project expense level calculations on form HUD-52723); 

• include more PHAs that have conducted an EPC by working with PHAs and the 

HUD Energy Center as well as using survey data collected to fill in EPC year 

(current data gap for many EPC PHAs);

• further analyze the impact of multiple EPCs conducted at a PHA; 

• expand UEL analysis to PHAs not included in the survey sample; 

• evaluate how outliers are excluded. For example, outliers were excluded at the 

PHA level, further analysis may be possible if outliers are instead excluded at the 

AMP level; and

• if available, obtain gross square footage data about PHAs to perform analysis 

based on energy use intensity.

D-1 

Appendix D 

Maps of Regions and of PHA Populations
D-2 

Note: There was no PHA selected from Alaska in the sample.
E-1 

Appendix E 

Abbreviations 

AMP Asset Management Project

CFR Code of Federal Regulations

EPC energy performance contract

ESCo energy service contractor

ESPC energy service performance contract

FASS Financial Assessment Subsystem 

FEMP Federal Energy Management Program

HUD U.S. Department of Housing and Urban Development

IQR inter-quartile range

LIHTC low-income housing tax credit

MASS Management Assessment Subsystem

O&M operating and maintenance 

PASS Physical Assessment Subsystem 

PHA public housing authority

PHAS Public Housing Assessment System

P.L. Public Law

QAP Qualified Allocation Plan

RAD Rental Assistance Demonstration

REAC Real Estate Assessment Center

RPU Residents-Paid Utility Incentive program

RR response rate

SD standard deviation

SIR savings-to-investment ratio

UEL Utility Expense Level
R-1 

References

Bordenave, J., R. Santangelo, S. Morgan, and M. Nail. 2016. The Intersection of EPC 

and RAD: A Roadmap for PHAS. Journal of Housing & Community Development 73 (2): 

27–34. https://ssfengineers.com/wp-content/uploads/2016/04/JHCDMarApr16-3-28-16-

Pivotal-Point-Apartments-EHA-ARC.pdf. 

U.S. Department of Housing and Urban Development (HUD). 2018a. Rental Assistance 

Demonstration. Washington, DC: HUD. 

https://www.hud.gov/sites/dfiles/Housing/documents/RAD_100000_Homes_20180813.pdf. 

———. 2018b. RAD Talk. Washington, DC: HUD. RAD in Review. 

https://www.hud.gov/sites/dfiles/Housing/documents/RAD_Jan2018_Newsletter_013118.pdf. 

———. 2018c. Notice PIH 2018-20—Partnering with Utility Companies on Energy 

Performance Contracts. Washington, DC: HUD. 

https://www.hud.gov/sites/dfiles/PIH/documents/PIH_2018-20_UPP_Notice.pdf. 

———. 2016. Interim Report—Evaluation of HUD’s Rental Assistance Demonstration. 

Washington, DC: HUD’s Office of Recapitalization: iii. 

https://www.huduser.gov/portal/sites/default/files/pdf/RAD-InterimRpt.pdf. 
UEL Data Analysis

C-3