Energy Evaluation of a New 

Construction Pilot Community: 

Fresno, California 

A. Burdick, A. Poerschke, A. Rapport, 

and M. Wayne

IBACOS, Inc.

June 2014
NOTICE

This report was prepared as an account of work sponsored by an agency of the United States government. 

Neither the United States government nor any agency thereof, nor any of their employees, subcontractors, or 

affiliated partners makes any warranty, express or implied, or assumes any legal liability or responsibility for the 

accuracy, completeness, or usefulness of any information, apparatus, product, or process disclosed, or 

represents that its use would not infringe privately owned rights. Reference herein to any specific commercial 

product, process, or service by trade name, trademark, manufacturer, or otherwise does not necessarily 

constitute or imply its endorsement, recommendation, or favoring by the United States government or any agency 

thereof. The views and opinions of authors expressed herein do not necessarily state or reflect those of the 

United States government or any agency thereof.

Available electronically at http://www.osti.gov/scitech

Available for a processing fee to U.S. Department of Energy

and its contractors, in paper, from:

U.S. Department of Energy

Office of Scientific and Technical Information

P.O. Box 62

Oak Ridge, TN 37831-0062

phone: 865.576.8401

fax: 865.576.5728

email: mailto:reports@adonis.osti.gov

Available for sale to the public, in paper, from:

U.S. Department of Commerce

National Technical Information Service

5285 Port Royal Road

Springfield, VA 22161

phone: 800.553.6847

fax: 703.605.6900

email: orders@ntis.fedworld.gov

online ordering: http://www.ntis.gov/ordering.htm

Printed on paper containing at least 50% wastepaper, including 20% postconsumer waste
iii

Energy Evaluation of a New Construction Pilot Community: 

Fresno, California

Prepared for: 

The National Renewable Energy Laboratory 

On behalf of the U.S. Department of Energy’s Building America Program 

Office of Energy Efficiency and Renewable Energy 

15013 Denver West Parkway 

Golden, CO 80401 

NREL Contract No. DE-AC36-08GO28308 

Prepared by: 

A. Burdick, A. Poerschke, A. Rapport, and M. Wayne 

IBACOS, Inc.

2214 Liberty Avenue 

Pittsburgh, Pennsylvania 15222

NREL Technical Monitor: Michael Gestwick

Prepared under Subcontract No. KNDJ-0-40341-03 

June 2014 
iv

[This page left blank]
v 

Contents

List of Figures ............................................................................................................................................ vi

List of Tables .............................................................................................................................................. vi

Definitions.................................................................................................................................................. vii

Executive Summary................................................................................................................................. viii

1 Introduction and Background ............................................................................................................. 1

1.1 House Characterization ........................................................................................................2

1.2 Variations from the Systems Integrated Measures Package ................................................4

2 Modeling Methods ................................................................................................................................ 6

2.1 Energy Modeling .................................................................................................................9

3 Research and Experimental Methods .............................................................................................. 10

3.1 Utility Data Collection .......................................................................................................10

3.2 Analysis Methodology .......................................................................................................10

4 Results................................................................................................................................................. 12

4.1 Annual Energy Use ............................................................................................................12

4.2 House 1 Utility Usage ........................................................................................................13

4.3 House 2 Utility Usage ........................................................................................................14

4.4 House 3 Utility Usage ........................................................................................................15

4.5 House 5 Utility Usage ........................................................................................................16

4.6 House 6 Utility Usage ........................................................................................................17

4.7 Inverse Modeling and Utility Bill Disaggregation .............................................................18

5 Discussion........................................................................................................................................... 20

5.1 Verification of Predicted Energy Savings ..........................................................................20

5.2 Disaggregated Load Analysis ............................................................................................20

5.3 Construction Best Practices ...............................................................................................21

5.4 Additional System Trade-offs ............................................................................................21

5.5 Construction Efficiencies Realized by Trade Partners ......................................................21

6 Conclusions ........................................................................................................................................ 23

References................................................................................................................................................. 24
vi 

List of Figures

Figure 1. Least-cost curve.......................................................................................................................... 6

Figure 2. Annual energy use comparison of utility bills and models.................................................. 12

Figure 3. House 1 gas usage.................................................................................................................... 14

Figure 4. House 1 electricity usage......................................................................................................... 14

Figure 5. House 2 gas usage.................................................................................................................... 15

Figure 6. House 2 electricity usage......................................................................................................... 15

Figure 7. House 3 gas usage.................................................................................................................... 16

Figure 8. House 3 electricity usage......................................................................................................... 16

Figure 9. House 5 gas usage.................................................................................................................... 17

Figure 10. House 5 electricity usage....................................................................................................... 17

Figure 11. House 6 gas usage ................................................................................................................. 18

Figure 12. House 6 electricity usage....................................................................................................... 18

Unless otherwise noted, all figures and photos were created by IBACOS. 

List of Tables 

Table 1. Plan Types..................................................................................................................................... 3

Table 2. Installed Options for Each Test House ...................................................................................... 4

Table 3. Summary of Occupant Self-Reported Occupancy and Miscellaneous Loads ....................... 4

Table 4. Duct Leakage to Outdoors and Whole-House Infiltration Rates ............................................. 5

Table 5. Annual Source Energy Savings for the Test House ................................................................. 8

Table 6. Adjusted Set Point Parameters................................................................................................. 13

Table 7. Change in Model Accuracy Due to Adjusted Set Points ........................................................ 13

Table 8. Disaggregated Energy Consumption from Inverse Modeling Compared to BEopt ............ 19

Unless otherwise noted, all tables were created by IBACOS.
vii 

Definitions

ACH50 air changes per hour at 50 Pascals

AFUE annual fuel utilization efficiency

AMY Actual Meteorological Year

BEoptTM Building Energy Optimization (software)

CAHP California Advanced Homes Program

CEC California Energy Commission

CFM cubic feet per minute

HSP House Simulation Protocols

IMT ASHRAE Inverse Modeling Toolkit 

JR/JC job ready/job complete

kWh kilowatt hours

SEER seasonal energy efficiency ratio

SIMP systems integrated measures package

TMY Typical Meteorological Year

WCHH Wathen Castanos Hybrid Homes, Inc.
viii

Executive Summary

To evaluate the success of implementing energy efficiency upgrades at the community scale 

using a whole-house systems integrated measures package (SIMP), IBACOS developed a field 

test plan for a new construction pilot community located in Fresno, California (mixed-dry 

climate). This pilot community project followed research conducted at the single occupied test 

house level with builder Wathen Castanos Hybrid Homes, Inc. (WCHH) to meet a 30% energy 

savings level with respect to Building America House Simulation Protocols (Hendron and 

Engebrecht 2010). The research was designed to evaluate and document the success of WCHH 

in applying the SIMP established in the single occupied test house to the production scale. 

Following the research results from the individual test house, the project evaluated the success of 

applying an energy-efficient SIMP at the community scale by addressing factors of 

implementation and scalability at the community scale. This evaluation included validation of 

modeled energy performance using collected utility bills for five actual houses, as well as shortterm performance testing (whole-house infiltration testing and duct leakage testing) and selfreported assessments from the builder and trade partners regarding implementation challenges 

and successes. The team implemented a low-cost data logger solution that was capable of being 

installed by the builder’s staff adjacent to each house’s thermostat. Using these data, the team 

was able to improve the accuracy of the models by up to 21% by creating set point schedules, 

which accounted for any anomalous occupant control behavior. The team performed a regression 

analysis on the utility bills to disaggregate heating and cooling energy usage from base load and 

compared the results to modeled heating and cooling energy consumption. 

Fresno, California, is a competitive market, and builders are seeking ways to differentiate 

themselves. The builder acknowledges that state and utility incentives continue to help pay for 

additional features in the test house, allowing the builder to add more energy-efficient features 

than it typically would include in its houses. This builder has demonstrated commitment to 

building high-quality, energy-efficient houses and has become a leader in its local market. 

However, without the available financial incentives, the builder would be unable to build to as 

high of a standard of energy efficiency as it currently builds. 

Results of monitoring and analysis efforts showed that, in general, the actual gas and electric 

usage was close to the predicted usage. A gap still existed between the Typical Meteorological 

Year and Actual Meteorological Year models for gas usage in the winter months. This gap is 

largely due to a 15% decrease in heating degree days for the actual weather year. 
1 

1 Introduction and Background

Builders face several key problems when implementing a whole-house systems integrated 

measures package (SIMP) from a single test house into multiple houses. Although a technical 

solution already may have been evaluated and validated in an individual test house, the potential 

exists for constructability failures at the community scale. Constructability failures can lead to 

performance issues such as higher energy use in the houses, durability issues such as moisture 

intrusion and building failure, and cost overruns. 

The intent of this research was to analyze the effectiveness of the SIMP to achieve the target 

energy savings predicted using Building Energy Optimization (BEoptTM) software, version 1.2. 

Key research for this project evaluated the consistency of executed construction specifications, 

pricing, and efficiencies of scale. Research focused on the builder and trade implementation of a 

SIMP and the actual utility usage in the houses at the community scale of production. The 

following research questions target the anticipated discrepancy between the predicted and actual 

energy consumption of the test houses: 

• How do the actual utility bill readings compare to projected energy consumption using 

BEopt when actual weather and thermostat set points are normalized? What factors 

account for any significant differences? 

• How closely does the normalized predicted heating and cooling energy usage align with 

actual usage when base load energy usage (water heating, lighting, appliances, and 

miscellaneous loads) is not included?

With any predictive energy modeling, there will always be gaps between the modeled inputs and 

the actual conditions of the house or houses that are being modeled. To produce the model, 

assumptions are made about actual weather conditions and operating conditions (i.e., occupant 

behavior) of the house. All predictive modeling seeks to accurately represent the actual energy 

usage of the house, and more accurate inputs should result in more accurate outputs. In this 

study, physical inspections of the houses and short-term performance testing (enclosure air 

leakage and duct air leakage tests) have validated many of the modeled inputs. The occupants 

completed evaluation forms to provide information on occupancy, lighting, and plug loads. For 

this current research, additional measurements on indoor and outdoor weather conditions will be 

monitored and will provide additional opportunities to adjust the model inputs and to normalize 

the outputs to actual weather conditions. 

This report addresses factors of implementation and scalability at the community scale and 

proposes methodologies by which community-scale energy evaluations can be performed based 

on results at the occupied test house level. Short-term testing results (duct air leakage testing, air 

infiltration testing) at the community scale are compared with specified targets. The construction 

efficiencies realized by trade partners that lead to reduced production costs at a community scale 

are discussed, along with other system trade-offs that can be implemented to capture additional 

energy savings at no extra upfront cost to the builder. 

Actual utility bill readings for five occupied houses are compared to projected energy 

consumption from models created using the BEopt software to quantify gaps in energy 
2 

performance between the models and the actual houses. Monthly energy use profiles compare 

predicted to actual heating, cooling, and base load (water heating and lighting, appliances, and 

miscellaneous loads) energy use. 

This new construction pilot community was constructed by builder-partner Wathen Castanos 

Hybrid Homes, Inc. (WCHH). In 2009 and 2010, IBACOS supported WCHH (Wathen-Castanos 

Homebuilders at the time) in the design and construction of an occupied test house that was to 

achieve greater than 30% energy savings with respect to the Building America House Simulation 

Protocols (HSP) (Hendron and Engebrecht 2010). The success of this single test house project 

was measured through an in-house data acquisition system and the results of a study presented in 

an earlier report published through Building America (IBACOS 2010). 

Nine occupied houses, comprising seven floor plans, initially were identified to be evaluated for 

this community-scale effort. Through foreclosure, vacancy, and other instances of attrition, the 

final house count for evaluation ended up at five houses comprising five floor plans. This 

evaluation included validation of modeled energy performance using collected utility bills for 

five occupied houses, as well as short-term performance testing (whole-house infiltration testing 

and duct leakage testing) and self-reported assessments from the builder and trade partners 

regarding implementation challenges and successes. 

Five occupants participated in this community-scale research by providing utility bills and 

information on occupancy and miscellaneous gas and electric appliance use for their houses. 

IBACOS used these utility data, measured thermostat temperature data, and background house 

information to analyze the actual energy performance of the houses and to help improve the 

accuracy of the original predictive modeling for these houses.

Verification with measured data is an important component in predictive energy modeling. The 

utility bill data confirmed that the as-built and as-occupied houses typically met the predicted

energy savings. 

To keep the scope of the project to something that could be implemented on a community scale, 

house electric and gas consumption was not submetered. To separate the base load from heating 

and cooling energy consumption, the team used the ASHRAE Inverse Modeling Toolkit (IMT), 

version 1.9 (Kissock 2002). 

1.1 House Characterization

Table 1 provides basic information to characterize each of the five test houses.
3 

Table 1. Plan Types

House

Number

House 

Model 

Number

Number 

of 

Stories

Size 

(ft2

)

Number 

of 

Bedrooms

Number 

of 

Bathrooms

Reported 

Number of 

Occupants

Date 

Construction 

Completed

1 202 1 2,028 4 2 2 12/16/2011

2 161 1 1,613 3 2 2 12/12/2011

3 178 1 1,788 4 2 2 8/26/2011

5 220 2 2,202 5 3 2 9/30/2011

6 220 1 1,435 3 2 2 10/27/2011

The standard SIMP for all five houses includes the following specifications:

• Foundation is uninsulated concrete slab on grade. 

• Stucco-clad exterior walls consist of 2 × 4 wooden framing with studs at 16 in. on center, 

insulated with R-13 fiberglass batts inside the wall cavities and 1-in. expanded 

polystyrene foam sheathing (R-4) on the exterior face, resulting in R-17 nominal thermal 

performance. Band joists are insulated with R-13 unfaced fiberglass batts.

• Knee walls are 2 × 4 wooden framing with studs at 24 in. on center, with R-13 fiberglass 

batt insulation inside the wall cavities and housewrap air barrier.

• The attic is insulated at the ceiling plane with loose-fill fiberglass to provide R-38 

nominal thermal performance. A radiant barrier is installed on the underside of the roof 

deck. 

• Windows are vinyl-framed, double-paned units with low-emissivity coatings and are 

argon filled, providing an overall heat transfer coefficient of U-0.34 and solar heat gain 

coefficient of 0.25. 

• The space conditioning system consists of a 94% annual fuel utilization efficiency 

(AFUE) natural gas furnace and a 16 seasonal energy efficiency ratio (SEER) airconditioning unit. 

• Domestic water heating is provided via a natural-gas-fired tankless unit with an energy 

factor of 0.82. Low-flow faucet aerators and showerheads are installed on all units.

• Ductwork is insulated to R-8 and is located in the unconditioned, vented attic along with 

the furnace. The total duct leakage to outdoors target is less than 5% of the system 

airflow.

• The target building enclosure airtightness level is 2.5 air changes per hour at 50 Pascal 

(ACH50) test pressure.

• Mechanical ventilation consists of timed exhaust fans with pressure relief. 

• All lighting (100%) installed at the interior and exterior of the house, including the 

garage, is high efficacy—using either light-emitting diodes or compact fluorescent lamps. 

• Major appliances (dishwasher, clothes washer, and refrigerator) are ENERGY STAR®

compliant. 
4 

1.2 Variations From the Systems Integrated Measures Package

All houses included in this study were constructed to the same standard SIMP; however, some of 

the units included options that were installed during or after construction. Table 2 highlights the 

significant options to the standard SIMP that were installed in each of the five test houses. 

Table 2. Installed Options for Each Test House

House 

Number

Thermal 

Enclosure

Space 

Conditioning Appliances

1 

R-49 attic insulation; R-13 Johns 

Manville Spider insulation in walls; 

U-0.33 windows 

Electric dryer

2 

R-49 attic insulation; R-13 Johns 

Manville Spider insulation in walls; 

U-0.33 windows 

Electric dryer

3 

R-49 attic insulation; R-13 Johns 

Manville Spider insulation in walls; 

U-0.33 windows 

Gas dryer

5 

R-49 attic insulation; R-13 Johns 

Manville Spider insulation in walls; 

U-0.33 windows 

19 SEER air 

conditioner Electric dryer

6 R-49 attic insulation; 

U-0.33 windows Electric dryer

Table 3 summarizes the catalog of miscellaneous electric loads provided by the occupants for the 

five test houses. This information was useful in performing the analysis on the data collected for 

this project. 

Table 3. Summary of Occupant Self-Reported Occupancy and Miscellaneous Loads

House

Number Occupancy TVs Computers Refrigerators

1 2 adults 4 flat screen 3 2

2 2 adults 3 liquid crystal 

display/flat screen 1 1

3 2 adults 1 flat screen,

1 tube type

1 laptop,

1 desktop 1

5 2 adults 1 flat screen (3D) 3 3 refrigerators, 

1 freezer

6 2 adults 1 liquid crystal display,

1 plasma 1 laptop 1 
5 

Finally, whole-house infiltration and duct leakage test results were collected from the local 

Home Energy Rating System Rater who previously conducted these tests for the builder. 

Table 4 lists the results of the whole-house infiltration and duct leakage tests for the five test

houses. 

Table 4. Duct Leakage to Outdoors and Whole-House Infiltration Rates

House

Number

Year 

Constructed

Duct Leakage to Outdoors 

(% of Total System Airflow)

Whole-House Infiltration 

(ACH50)

Target – 5% 2.50

1 2011 4% 3.04

2 2011 6% 3.49

3 2011 5% 3.04

5 2011 4% 2.35

6 2011 5% 2.95
6 

2 Modeling Methods

The builder was largely interested in the salability of the SIMP included in the original occupied 

test house that was constructed in 2009–2010 by the builder. Fresno, California, is a competitive 

housing market, and builders are seeking ways to differentiate themselves from their 

competitors. This builder acknowledged that state and utility financial incentives helped to pay 

for some of the additional energy-efficient features in the test house, allowing the builder to add 

more energy-efficient features than would typically be included. 

Optimization of the building enclosure and mechanical systems was performed using BEopt 

version 2.0.0.4 and resulted in a least-cost curve highlighting several key upgrades to the 

builder’s standard specification package. Figure 1 shows the least-cost curve with one of the 

optimized specification packages selected. 

Figure 1. Least-cost curve

Many of the optimized specifications identified by BEopt were selected by the builder, with the 

following exceptions: R-19 insulated wall with 2 × 6, 24-in. on center framing, SEER 15 air 

conditioner, 98% AFUE natural gas furnace, and ducts in finished space. These specifications 

were replaced with other specifications based on availability, available incentives, appeal to 

customers, and local building practices. 

The original test house SIMP was selected to be cost neutral to the consumer over an amortized 

30-year mortgage at 7% interest; however, the original test house project did not have a costneutral first cost to the builder, resulting in an increased sale price to the consumer for that 

house. To lower the sale price for future houses, the builder adopted a newer baseline measures 

package that included most of the specifications from the 2009–2010 test house plus a few 

lower-tier modifications such as a reduced SEER rating on air-conditioning equipment, a 

reduced energy factor water heater, and reduced R-value of attic insulation from R-44 to R-38. 
7 

IBACOS conducted extensive modeling on the original 2010 test house design by using 

EnergyGauge USA (EnergyGauge 2012) to zero in on a specification package that met the target 

performance level, fit the builder’s marketing agenda, and addressed limited product availability 

or trade experience. IBACOS recommended that the new design incorporate a “ducts inside 

conditioned space” strategy to improve the performance of the space conditioning system. 

However, the builder felt that its signature 10-ft ceilings were an essential part of its successful 

marketing program and did not want any dropped soffits or bulkheads to interfere with the 

ceiling plane. The builder also considered a cathedralized attic strategy, but the pricing did not fit 

the budget. Ultimately, the builder achieved greater than 50% source energy savings by

enhancing the thermal enclosure and installing the highest performance gas-fired heating 

equipment and cooling equipment it could acquire through its preferred supplier.

The final specification package for the initial occupied test house included energy features that 

achieved Title 24 (2008 Standards) of the California Energy Commission (CEC 2008), met the 

California Advanced Homes Program (CAHP) Tier II qualifications (2010–2012 Program) 

(CAHP 2012), and reached 52% whole-house energy savings compared to the 2009 Building 

America Benchmark (Hendron and Engebrecht 2009). 

The most significant upgrades that contributed to this level of savings included a 94.7% AFUE

furnace, 19 SEER air conditioner, and 0.98 energy factor tankless water heater. The total duct 

leakage was measured at 5% of total fan flow. Total building leakage was measured at 2.7 

ACH50. Reduced air leakage to the building enclosure was achieved by employing an airtight 

drywall strategy, including gluing the drywall to the wood framing around the perimeter of each 

drywall panel. Whole-house ventilation was provided using two Broan SmartSense fans, each 

rated to 80 CFM, working intermittently and in unison to provide the equivalent of 58 CFM of 

continuous exhaust from two bathroom locations. Ventilation exhaust rates were set to comply 

with ASHRAE Standard 62.2 (ASHRAE 2010). Elevated insulation levels in the walls (R-19) 

were achieved using Johns Manville Spider blown-in fiberglass cavity fill combined with R-4 

expanded polystyrene continuous foam sheathing on the exterior. The ceiling was insulated with

blown-in fiberglass that achieved R-49. 

The builder met the cost-neutrality goal for the original, single test house. IBACOS used BEopt 

to calculate the annual cost savings for this house plan. The annual cost savings due to energy 

efficiency improvements (without photovoltaics) were predicted to be $1,612, and the annual 

increase in mortgage payments (30-year term, 5% annual interest rate1

) to finance the 

improvements was predicted to be $112, leading to a net annual cost savings to the occupant of 

$1,500/year. The 2010 utility rates for Clovis, California, where the community is located, were 

$0.13/kWh and $1.80/therm. 

Table 5 shows the annual source energy consumption and savings and the annual utility bill 

reduction of the 2009 Building America Benchmark (Hendron and Engebrecht 2009) compared 

to the test house. Incremental costs (total and amortized annual) also are shown for the test house 

compared to the builder’s standard house at the time (circa 2009–2010). 

 

1 U.S. Department of Energy, “Methodology for Evaluating Cost-Effectiveness of Residential Energy Code 

Changes.” http://www.energycodes.gov/sites/default/files/documents/residential_methodology.pdf. 
8 

Table 5. Annual Source Energy Savings for the Test House

Description of 

End Use

Annual 

Source Energy

Estimated 

Source Energy Annual Utility Bill 

Reduction

Test House with 

Respect to

Benchmark

Incremental Costs

Test House with Respect to 

Builder’s Standard

Benchmark

(MBtu/

yr)

Test 

House

(MBtu/

yr)

Percent of 

End Use

Versus

Benchmark

Percent of 

Total 

Versus

Benchmark

Total Builder 

Cost + 10% 

Markup

Amortized Annual 

Cost 30-Year 

Mortgage, 

7% Interest

Space Heating 64 30 54% 14% $553 $1,364

$864 thermal 

enclosure + $500 

space 

conditioning 

$88

$56 thermal 

enclosure + $32 

space 

conditioning 

Space Cooling 69 21 70% 19% $553 

Domestic 

Water Heating 22 12 44% 4% $160 $350 $23 

Lighting 29 13 54% 6% $174 $20 $1

Appliances and

Miscellaneous 

Electric Loads

57 43 24% 6% $158 $0 $0 

Outdoor Air 

Ventilation 3 2 40% 1% $15 $0 $0 

Subtotal 252 121 52% 52% $1,612 $1,734 $112

Photovoltaic

Panel 

Generation

0 –41 NA 17% $464 $29, 358 $1,891 

Total 252 121 52% 52% $2,076 $31,092 $2,003
9 

2.1 Energy Modeling

The time frame of this project from individual test house to community-scale evaluation 

occurred over a 31⁄2-year period from 2009 to 2013. During this period, changes to the Building 

America benchmark and energy modeling applications resulted in a number of revisions to the 

analysis. Initial target energy savings were measured against the earlier version of the Building 

America Benchmark house (Hendron and Engebrecht 2009). Under that 2009 Building America 

Benchmark, the test house was projected to achieve a minimum source energy savings of 50% 

over that target. As the project moved into the community scale, energy savings were predicted

using BEopt version 1.0.1 against the HSP (Hendron and Engebrecht 2010), with the original 

SIMP projected to achieve a minimum source energy savings of 30% over the target. This latest 

community-scale effort builds off the measured energy savings of the single test house, with 

projected energy savings of 30% relative to the HSP using BEopt E+ version 1.2 (BEopt E+). In 

addition, all test houses in this community were constructed in accordance with Pacific Gas & 

Electric’s CAHP, which pays incentives to builders based on the level of energy efficiency that 

they achieve. WCHH is participating at the Tier II level (30% better than California Code of 

Regulations, Title 24) (CEC 2008). 
10 

3 Research and Experimental Methods

To properly characterize the five test houses included in this community-scale study, IBACOS 

collected data from the builder on the “as-built” conditions of the houses and used that data to 

inform the predictive energy models that were created using BEopt version 2.0.0.4. The research 

team then compared the energy usage produced from the models to that collected from the utility 

bills of the houses, enabling the team to identify the possible causes of any differences in 

consumption. IBACOS also compared the modeled energy use and usage based on the utility 

bills with the predicted energy use of the Building America Benchmark (Hendron and 

Engebrecht 2009) and the original test house SIMP. 

3.1 Utility Data Collection

IBACOS collected data on whole-house energy consumption by obtaining occupant utility bills 

for both electricity and natural gas use for each test house. In addition, indoor temperature 

measurements were taken in each test house at the site of the thermostat using a single Onset 

HOBO2 at each location. Due to the fragmented nature of the utility bills, whenever a full 

month’s worth of data was unavailable, the team extrapolated those data to fit the entire month.

Additionally, a weather station was installed in a central location among the test houses to collect 

data on outdoor temperature, wind speed, and solar irradiance via a single HOBO data logger. 

Because this station did not have a remote connection, a failure in the logger that stopped data 

collection after several days was not discovered until the logger was retrieved at the end of the 

analysis period. As a surrogate for measured data, historical weather data were purchased as an 

Actual Meteorological Year (AMY) data file from Weather Analytics LLC.3 To maximize 

accuracy, data from the nearest station, Fresno Air Terminal, was chosen. Weather Analytics 

uses a set of algorithms to fill and smooth missing data, allowing a continuous EnergyPlus 

weather file to be generated for the period of monitoring (EnergyPlus 2012). The team then ran a 

BEopt model for each house using the AMY file to normalize for weather conditions. 

3.2 Analysis Methodology

Traditionally, linear regression is used to create a model of a building’s energy consumption as it 

responds to outdoor temperature changes. Utility bills are then normalized to the Typical 

Meteorological Year 3 (TMY3) outdoor temperature data used in an energy model for 

comparison. Instead, the team chose to use an AMY weather file to normalize modeled energy 

consumption to utility bills so that additional errors would not be introduced from the process of 

linear regression and to allow monthly energy consumption to be compared. Additionally, using 

an AMY file allows all meteorological variables, beyond dry bulb temperature, to be normalized 

in a robust fashion. 

Set point and set back parameters were determined by visually inspecting the HOBO thermostat 

measurements. The team determined set back schedules by selecting days that best matched 

typical behavior. If no regular set back schedule was observed, the team assumed a constant set 

point value representative of average behavior. 

 

2 Onset Computer Corporation. http://www.onsetcomp.com/products/data-loggers. 3 Weather Analytics LLC. http://www.weatheranalytics.com/. 
11 

Inputs to the BEopt energy models included the standard specifications (Hendron and 

Engebrecht 2009) from the original SIMP along with the installed options for each house using 

both the standard temperature set points and the actual set points measured by the HOBOs, and 

the measured test results for whole-house infiltration and duct leakage. 
12 

4 Results 

This section provides details about the annual and monthly profiles of the actual (utility bills) 

and modeled energy consumption for the five test houses. 

4.1 Annual Energy Use

Figure 2 summarizes the modeled energy use compared to the actual 12-month utility bill data

for each of the five test houses. It shows that the actual energy use generally was less than the 

modeled energy use for each test house, with the exception of House 5. The differences among 

the various models and actual usage varied for the different houses. However, because few 

changes were made to the as-built houses from the SIMP, the energy use for those two modeled 

scenarios remained fairly close.

Figure 2. Annual energy use comparison of utility bills and models

Using temperature measurements taken at the thermostat over the course of one year, the team 

was able to determine an adjusted set point and set back schedule, as shown in Table 6. Using 

this information, the team adjusted the BEopt models to reflect occupant behavior. Table 7

summarizes the results from this modification. 

0.0

50.0

100.0

150.0

200.0

Benchmark

SIMP

As Built

Utility

Benchmark

SIMP

As Built

Utility

Benchmark

SIMP

As Built

Utility

Benchmark

SIMP

As Built

Utility

Benchmark

SIMP

As Built

Utility

Home 1 Home 2 Home 3 Home 5 Home 6

Energy Usage (Mbtu)

Electric

Gas
13 

Table 6. Adjusted Set Point Parameters

House 1 House 2 House 3 House 5 House 6

HeatingSet Point 74° F 73° F 68° F 71° F 70° F

Set Back 70° F 70° F 64° F 68° F –

Schedule 00:00 to 10:00 00:00 to 17:00 00:00 to 10:00 05:00 to 18:00 –

Cooling Set Point 80° F 75° F 80° F 79° F 70° F

Set Back – – – – 75° F

Schedule – – – – 08:00 to 22:00

Table 7. Change in Model Accuracy Due to Adjusted Set Points 

House 

1

House 

2

House 

3

House 

5

House 

6

Annual Energy 

Consumption 

(MMBtu/Yr)

Standard set 

points* 134 112 110 142 107 

Adjusted set points 125 106 92 132 111

Utility bill 90 85 83 140 104

Percent change 7.4% 5.8% 19.4% 7.6% –3.8%

A standard set back schedule was observed: the team assumed a constant set point value representative of 

average behavior.

*Standard set points: 71°F heating, 76°F cooling (Hendron and Engebrecht 2010). 

4.2 House 1 Utility Usage

Figure 3 and Figure 4 show gas and electricity usage per month for House 1, respectively, from 

June 2012 through May 2013. The graphs, which have been wrapped to show a normal calendar 

year, include energy usage data gathered from the TMY model, AMY model, and utility bills.

There were a few gaps in the utility bill data for May, June, and December because of collection 

difficulties. This was accounted for in these graphs using the days of that month with data to 

average a monthly energy usage value. As the two figures show, the gas usage was relatively 

close among the two models and actual data, but a significant difference occurred with electricity

usage. The actual usage was much lower than that predicted by the model, which is most likely 

because of occupant behavior. 
14 

Figure 3. House 1 gas usage 

Figure 4. House 1 electricity usage 

4.3 House 2 Utility Usage

Similarly, Figure 5 and Figure 6 show the gas and electricity usage per month, respectively, for 

House 2 from January 2012 through December 2012. The graphs include energy usage data 

gathered from the TMY model, AMY model, and utility bills. Partially missing utility data in 

January, March, and April were accounted for as previously stated. The results for House 2 are 

similar to those for House 1; the gas usage was fairly close, but a gap exists between modeled 

and actual electricity usage. Also, a clear difference between the TMY and AMY models exists 

for gas usage in the winter months, which can be attributed to 15% fewer heating degree days for 

the actual weather year. 
15 

Figure 5. House 2 gas usage

Figure 6. House 2 electricity usage 

4.4 House 3 Utility Usage

The gas and electricity usage per month from the TMY model, AMY model, and utility bill data 

for House 3 are shown in Figure 7 and Figure 8, respectively, from January 2012 through 

December 2012. No gaps in the utility data existed for this house, making the data as accurate as 

possible. In general, the actual gas and electricity usage was close to the predicted usage. A gap 

still existed between the TMY and AMY models for gas usage in the winter months. This gap is 

largely due to a 15% decrease in heating degree days for the actual weather year. 
16 

Figure 7. House 3 gas usage

Figure 8. House 3 electricity usage

4.5 House 5 Utility Usage

The projected and actual gas and electricity usage for House 5 is represented in Figure 9 and 

Figure 10, respectively, from February 2012 through January 2013. Graphs have been wrapped 

to show a normal calendar year.Partially missing utility bill data in February, June, and July were 

accounted for by the same method previously mentioned. Both models and the actual utility data 

were very similar over the year for gas and electricity usage, showing accurate prediction from 

the models. 
17 

Figure 9. House 5 gas usage

Figure 10. House 5 electricity usage

4.6 House 6 Utility Usage

Figure 11 and Figure 12 show gas and electricity usage per month for House 6, respectively, 

from February 2012 through January 2013. Graphs have been wrapped to show a normal 

calendar year. The included energy usage data were gathered from the TMY model, AMY 

model, and utility bills. Utility bill data partially missing in July and August were accounted for 

by the same method previously described. The gas usage is fairly similar among the two models 

and actual data, but a slight difference existed between the models and the actual data for 

electricity usage. 
18 

Figure 11. House 6 gas usage

Figure 12. House 6 electricity usage

4.7 Inverse Modeling and Utility Bill Disaggregation

One of the additional research questions the team considered was how well the predicted heating 

and cooling energy usage aligned with actual usage. Heating and cooling energy usage was not 

submetered as part of this project; therefore, regression analysis was necessary to estimate the 

base load. To perform this analysis, the team used the ASHRAE IMT, version 1.9 (Kissock 

2002). This tool takes utility bills or energy consumption measurements and performs a linear 

regression against weather data, typically outdoor temperature. The team decided to use a threeparameter regression on the monthly utility bills and hourly weather data, treating the gas and 

electric data separately to improve accuracy. The result of this regression is a heating change-
19 

point, a cooling change-point, and the slope of the heating and cooling lines, indicating how the 

house responds to colder or warmer outdoor temperatures. 

Table 8 summarizes the results of this regression. Total error of the regression model as indicated 

by the R2 value was minimal for all but one of the houses considered in the analysis. Despite the 

low error in the regression model, the disaggregated cooling and heating energy usage does not 

align well with BEopt predictions. The heating energy usage predicted from the IMT does not 

take into account fan energy; therefore, this was not summed from the BEopt output. A result of 

this is that the electrical base load also includes the heating fan energy, raising the apparent base 

load. 

Table 8. Disaggregated Energy Consumption from Inverse Modeling Compared to BEopt

 Cooling Energy Usage (kWh) Heating Energy Usage (Therms)

House 

Number BEopt Utility –IMT

Percent 

Difference* R2 BEopt Utility –IMT

Percent 

Difference* R2

1 970 1,549 –37% 0.97 233 126 85% 0.89

2 925 2,097 –56% 0.88 164 47 250% 0.97

3 442 1,200 –63% 0.80 81 46 75% 0.95

5 1,137 1,596 –29% 0.84 153 121 27% 0.95

6 846 2,096 –60% 0.63 192 71 171% 0.98

* Percent difference of BEopt results compared to the inverse model.
20 

5 Discussion 

This section presents the analysis of the measured and modeled data. As shown in Table 1, 

through occupant surveys, the team determined that each of the five test houses was underoccupied relative to the HSP (Hendron and Engebrecht 2010). As a result, the utility bill data 

showed less consumption than predicted. 

5.1 Verification of Predicted Energy Savings

Verification with measured data is an important component of predictive energy models, 

specifically when there is a target energy savings. Incorrectly installed or malfunctioning 

heating, ventilation, and air conditioning equipment can be problematic in the residential 

building industry. Additionally, many simplifications and assumptions must be made regarding 

the as-built and as-occupied house. 

To identify any inaccuracies in the modeling processes, the team implemented an inexpensive 

monitoring program through which the test house occupants submitted utility bills, and a HOBO 

data logger was installed next to the thermostat in each test house. This method allowed for the 

installation of hardware by an untrained layperson representing the builder. 

For all houses presented in this report, the team was able to estimate seasonal set points and set

back schedules, as well as to analyze any aberrant occupant thermostat control behavior. 

Reliance on the occupants’ diligence in submitting monthly utility bills increased the risk of 

inconsistencies or missing values in the data. Despite this deficiency, the utility bill data 

confirmed the as-built and as-occupied houses typically met the predicted energy savings. 

This method of verification proved useful and cost effective at the community scale. A more 

expensive monitoring system (more than $5,000) using a traditional data logger, sensors, and 

telemetry is an effective approach for a single house but is impractical for the community scale. 

Based on measurements taken at the thermostat, the team was able to improve the accuracy of 

the simulations by 7% to 21% in four of the five houses, with the accuracy of the fifth house 

reduced by 4%. Although this may not influence the design stage of the house construction, these 

data provide an indication that standard set point and occupancy schedules might not be 

appropriate to be uniformly applied during the decision-making process. A stochastic method of 

predicting occupant behavior would provide a range of possible energy savings. 

5.2 Disaggregated Load Analysis

Disaggregated heating and cooling energy consumption from the IMT aligned poorly with the 

values predicted by BEopt in this project. This is largely due to the way the IMT estimates 

coefficients during regression analysis. The tool attempts to minimize the error in total energy 

consumption, while not giving weight to how well base load is separated from cooling and 

heating energy. Low load homes exacerbate the discrepancy because of greater influence of base 

load relative to heating and cooling loads. Lower magnitude numbers also are more susceptible 

to noise. Using monthly utility bills alone is not a reliable means to estimate disaggregated 

heating and cooling energy usage. To determine how well BEopt predicts heating and cooling 

energy usage, it is necessary to submeter the equipment separately.
21 

5.3 Construction Best Practices

The builder uses a third-party inspection provider for high-level liability issues such as framing 

and water intrusion issues. The third-party program inspection occurs at the rough stages in 

every house after windows are installed. Quality assurance inspections of these key areas in 

every house are a big part of the builder’s ability to deliver consistent quality to the occupants. 

Random inspections of other structure areas, such as foundations, add to the overall quality 

assurance/quality control process. Defects are red tagged and are given a tracking number for 

reporting purposes. A lot-specific report then is sent to the site contact for resolution. The site 

contact manages resolution through the trade, and then both the trade and the site contact sign off 

that the repairs have been made properly. The third-party inspector then sends an aging report of 

open items to the builder’s office for follow-up and, ultimately, resolution. 

The builder also uses an in-house program tied to first-time quality, which it learned from its 

work with the National Housing Quality Award program. A series of checklists from first-time 

quality has been developed by the builder specifically for the builder’s practices. As of the time 

of completion of construction of the community, these checklists had been in place less than a 

year, and a few trades have adopted the practice while others have not. This is a “ground-up” or 

grass-roots effort to identify areas for quality inspection, and checklists are generated for each 

trade with the trade’s input. Each phase of work has a checklist for each associated trade. The 

trade foreman walks the house and rates the job on a job ready/job complete (JR/JC) form for the 

previous trade. The JR/JC forms are submitted to both the trade’s offices and the builder’s 

offices. The builder site superintendent also back checks the trade foreman with its own JR/JC 

form. The builder management analyzes the JR/JC forms for trends to identify hot spots and 

works with trades to address issues. 

Performance testing occurs on a sample of houses with blower door, duct tightness, refrigerant, 

maximum cooling, and soon to be required low-leakage tests. Infiltration testing results are 

consistently within the program limits, with only one in every 25 to 30 houses having an issue. A 

final duct blaster test is performed after carpets are installed in the house. The site superintendent 

is present in the house during the testing to strengthen all parties’ understanding of what is being 

tested.

5.4 Additional System Trade-offs

Relocating the tankless water heater to the attic to reduce the amount of wasted water is one 

system trade-off under consideration. The relocation of the tankless water heater to the attic also 

reduces the risk of theft from the house during construction, thereby reducing the builder’s risk. 

The builder would like to take advantage of the low-leakage air handler credit from the CEC 

when the equipment becomes available through its preferred manufacturer. The builder also is 

looking forward to the CEC certifying low-leakage air handlers for its current manufacturer, 

reducing the Title 24 duct leakage requirements (CEC 2008). 

5.5 Construction Efficiencies Realized by Trade Partners

One critical success factor identified by the builder was finding the right trade partner that was 

willing to work with and deal with the changes the SIMP involved. The tricky details of air 

sealing and installing insulation around pipes or the Title 24 caulking details were identified as 

challenging for trade partners to achieve. 
22 

IBACOS conducted a phone interview with the insulators to discuss the implementation of the 

SIMP at a community scale (Cooper 2012; Ewing 2012). The insulators perform the insulation 

and air sealing scope. To reduce their production costs and to minimize the costs associated with 

training new crews, the insulators try to keep the same crews going to the builder’s houses when 

possible. To bring a new crew member up to speed requires one week of on-the-job training by a 

production manager before the new crew member can be fully utilized. The insulators estimated 

a 15% administrative cost increase over their typical work associated with the additional labor 

and paperwork required for the builder’s work. 

The insulators viewed the builder’s quality assurance/quality control program as being similar to 

their own internal quality checks. They clearly saw the value in the process and indicated its 

connection to the Title 24 requirements. 

Although the insulation and air sealing scope are directly related to the performance of the house, 

there is not much interaction between the installer and tester except when a solution to a problem 

is needed. Sealing of outlet plates was given as an example of cooperative work between the two. 

The insulators worked with the builder to achieve the desired results in the fireplace air barrier 

details and are working to address a question on the use of raised heel trusses. Air sealing around 

fire sprinkler heads and the best attic insulation material for the money are other issues they are 

working together to solve. 

The insulators reaffirmed that this builder is one step ahead of other builders in the area in 

utilizing heating, ventilation, and air conditioning supplies sealed to surfaces, caulking plates, 

and interior wall sealing. They see other builders following this builder’s lead. Most of this 

builder’s practices are the insulators’ standard practices, although the insulators did not indicate 

whose practices came first. 
23 

6 Conclusions

The following research questions target the anticipated discrepancy between the predicted and 

actual energy consumption of the test houses: 

• How do the actual utility bill readings compare to projected energy consumption using 

BEopt? 

• How closely does the normalized predicted heating and cooling energy usage align with 

actual usage when base load energy usage (water heating, lighting, appliances, and 

miscellaneous loads) is not included?

Nine occupied houses, comprising seven floor plans, initially were identified to be evaluated for 

this community-scale effort. Through foreclosure, vacancy, and other instances of attrition, the 

final house count for evaluation ended up at five houses comprising five floor plans. 

Verification with measured data is an important component of predictive energy models. As 

shown in Section 5.1, the utility bill data confirmed that the as-built and as-occupied houses 

typically met the predicted energy savings. 

By measuring the temperature at the thermostat and estimating set points, the team was able to 

account for occupant behavior. The modified set points improved the accuracy of the simulations 

by 7% to 21% in four of the five houses, with the accuracy of the fifth house reduced by 4%. 

Using the IMT and monthly utility bills is not a reliable means to estimate disaggregated heating 

and cooling energy usage. To determine how well BEopt predicts heating and cooling energy 

usage, it would be necessary to submeter the equipment separately.

With continued implementation of quality assurance/quality control measures that were 

previously developed and executed by the builder, the results of performance testing in future 

houses in this community are likely to continue meeting the targeted levels of performance. 

With feedback from the trades and the builder regarding efficiencies gained through consistent 

execution of the SIMP and discussions around other possible measures, additional opportunities 

were identified for further improving the levels of energy efficiency that will bring the 

incremental cost of upgrades closer to being first-cost neutral to the builder. 
24 

References 

ASHRAE (2010). ANSI/ASHRAE Standard 62.2 – 2010. Ventilation and Acceptable Indoor Air 

Quality in Low-Rise Residential Buildings. Atlanta, GA: ASHRAE.

BEopt. Building Energy Optimization with Hour-by-Hour Simulations, Version 1.2. Golden, 

CO: National Renewable Energy Laboratory. 

BEopt E+. BEopt E+ 1.2 Building Energy Optimization software, Version 1.2. Golden, CO: 

National Renewable Energy Laboratory. 

CAHP (2012). California Advanced HomesTM Program, Tier II qualifications (2010–2012 

Program). Sacramento, CA: Pacific Gas and Electric Company. Accessed June 

2014. www.californiaadvancedhomes.com/about-cahp/financial-incentives. 

CEC (2008). California Code of Regulations, Title 24. 2008 Building Energy Efficiency 

Standards for Residential and Nonresidential Buildings. Sacramento, CA: California Energy 

Commission. 

Cooper, C. (2012). Personal communication. Bakersfield, CA: Broken Drum Insulation. 

EnergyGauge (2012). EnergyGauge USA home energy simulation software. Accessed June 

2014. www.energygauge.com/usares//.

EnergyPlus (2012). EnergyPlus energy simulation software. 

http://apps1.eere.energy.gov/buildings/energyplus/. 

Ewing, M. (2012). Personal communication. Bakersfield, CA: Broken Drum Insulation. 

Hendron, R.; Engebrecht, C. (2009). Building America Research Benchmark Definition. Golden, 

CO: National Renewable Energy Laboratory. 

Hendron, R.; Engebrecht, C. (2010). Building America House Simulation Protocols. Golden, 

CO: National Renewable Energy Laboratory. 

IBACOS (2010). Building America Final Technical Report. Pittsburgh, PA: IBACOS.

Kissock, J.K.; Haberl, J.S.; Claridge, D.E. (2002). Development of a Toolkit for Calculating 

Linear, Change-Point Linear and Multiple-Linear Inverse Building Energy Analysis Models.

Final Report, ASHRAE Research Project 1050-RP. Accessed June 2014. 

http://repository.tamu.edu/bitstream/handle/1969.1/2847/ESL-TR-02-11-02.pdf?sequence=1. 
DOE/GO-102014-4433 ▪ June 2014

Printed with a renewable-source ink on paper containing at 

least 50% wastepaper, including 10% post-consumer waste.

buildingamerica.gov