State-of-the-Art Review for Commissioning Low Energy Buildings:

 Existing Cost/Benefit and Persistence Methodologies and Data, State of

 Development of Automated Tools and Assessment of Needs for

 Commissioning ZEB 

Marti Frank, Hannah Friedman and Kristin Heinemeier

Portland Energy Conservation, Inc.

Cory Toole and David Claridge 

Texas A&M University 

Natascha Castro

National Institute of Standards and Technology 

Philip Haves

Lawrence Berkeley National Laboratory

NISTIR 7356

 

August 2007

State-of-the-Art Review for Commissioning Low Energy Buildings: 

Existing Cost/Benefit and Persistence Methodologies and Data, State of 

Development of Automated Tools and Assessment of Needs for 

Commissioning ZEB 

Marti Frank, Hannah Friedman and Kristin Heinemeier

Portland Energy Conservation, Inc.

Cory Toole and David Claridge 

Texas A&M University 

Natascha Castro

National Institute of Standards and Technology 

Philip Haves

Lawrence Berkeley National Laboratory

Prepared for:

The U.S. Department of Energy

Energy Efficiency and Renewable Energy

Office of Building Technology, State and Community Programs

U.S. Department of Commerce

Carlos M. Gutierrez, Secretary

National Institute of Standards and Technology

William A. Jeffrey, Director

NISTIR 7356
This work was supported by the Assistant Secretary for Energy Efficiency and 

Renewable Energy, Office of Building Technology, State and Community Programs of 

the U.S. Department of Energy under Contract No. DE-AC02-05CH11231. This work 

was carried out as part of the U.S. contribution to the International Energy 

Agency/Energy Conservation in Buildings and Community System’s research project: 

Annex 47 on Cost-effective Commissioning of Existing and Low Energy Buildings. 

 
Table of Contents 

Executive Summary ..............................................................................................1

Overview...............................................................................................................5

Chapter 1: Review of Commissioning Cost-Benefit Methodologies and Data......7

I. Introduction.............................................................................................................................. 7

II. Data Collection Strategies.................................................................................................... 11

III. Methodologies for Determining Costs, Energy Benefits and Non-Energy Benefits............ 18

IV. Recommendations, Decision Points, and Next Steps ......................................................... 29

References ................................................................................................................................ 31

Chapter 2: Review of Persistence of Commissioning Measures in New and 

Existing Buildings................................................................................................35

I. Introduction............................................................................................................................ 35

II. Persistence of Commissioning Measures in Existing Buildings........................................... 35

III. Persistence of Commissioning Measures in New Buildings ............................................... 49

IV. Strategies for Improving Persistence in New and Existing Buildings ................................. 53

V. Related Reports................................................................................................................... 55

VI. Methodologies for Determining Persistence of Commissioning Measures and Energy 

Benefits of Commissioning........................................................................................................ 56

VII. Summary and Conclusions................................................................................................. 57

References ................................................................................................................................ 58

Chapter 3: Review of Automated Commissioning Tools for Buildings ...............60

I. Introduction............................................................................................................................ 60

II. Automated Commissioning Tools ......................................................................................... 60

III. Related Research................................................................................................................ 67

IV. Market Penetration.............................................................................................................. 73

References ................................................................................................................................ 74

Chapter 4: Review of Needs and Challenges in Commissioning Zero Energy 

Buildings .............................................................................................................78

I. Introduction............................................................................................................................ 78

II. Literature Review ................................................................................................................. 79

III. Discussion........................................................................................................................... 83

IV. Conclusions and Recommendations .................................................................................. 84

References ................................................................................................................................ 85

Chapter 5: New Functional Tests for use in Commissioning Zero and Low 

Energy Buildings.................................................................................................87

1. Introduction....................................................................................................................... 87

2. Radiant Slab Functional Test ........................................................................................... 87

3. Underfloor Air Distribution Plenum Pressure Test............................................................ 87

4. Demand-Controlled Ventilation......................................................................................... 88

5. Building Pressurization..................................................................................................... 88

Conclusions ........................................................................................................89

Appendix: Automated Commissioning Tool Interfaces.......................................91

 
Executive Summary 

This report documents the findings of a review of the state of the art in the 

commissioning of low energy buildings. 

The purpose of this report is threefold: 

1) To identify and assess existing methodologies for defining the costs and benefits 

of commissioning, including the persistence of benefits; 

2) To review and assess the state of the art in automated and semi-automated 

commissioning tools; and 

3) To assess current practices for commissioning low energy buildings and identify 

the needs for methods and tools that go beyond what conventional commissioning 

approaches can offer. 

Specific R&D recommendations are included in each chapter of the report and a 

summary is included in the Conclusions on page 89. 

Chapter 1: Review of Commissioning Cost-Benefit Methodologies and Data 

This chapter assesses the cost and benefit methodologies employed in 11 commissioning 

studies. Among these studies, there is no standard methodology for determining costs and 

benefits and the reported cost and benefit figures vary widely. Cost and benefit 

methodologies have been categorized as simple, moderate, or complex based on their 

level of complexity. One commonality exists across cost, energy and non-energy benefit 

methodologies: only complex methodologies validate the data. 

It is apparent that the methodology significantly impacts reported costs and benefits. 

Commissioning costs trend upward as the cost methodology becomes more complex. This 

is likely because more cost categories are taken into account. Non-energy benefits, on the 

other hand, are reported as higher when a moderate methodology is employed. This is 

likely due to the difference between “avoided cost” calculations, used in moderate 

methodologies, and “willingness to pay” calculations, used in complex methodologies. 

Three recommendations were derived from this assessment of existing cost-benefit 

methodologies: 

• Building commissioning data should be greatly diversified. The pool of buildings 

from which data are drawn should be expanded, as most of the data in these studies 

come from a small pool of commissioning projects and providers. 

• The data collection strategy must be matched to the data requirements of the cost and 

benefit methodologies. Complex methodologies, and even moderate methodologies, 

should aim to collect data using an ongoing, rather than retrospective, procedure. One 

way to do this is to facilitate the ability of respondents to enter their own data in realtime. An automated or semi-automated analysis tool could also be used to facilitate 

ongoing analysis. 

• Data validation is an important aspect of any cost-benefit methodology. In cases 

where data are not verified, the accuracy of the cost-benefit results may be at risk. 

 1 
Chapter 2: Review of Persistence of Commissioning Measures in New and Existing 

Buildings 

This chapter provides an overview of persistence studies in both new and existing 

building commissioning projects. Interest in the persistence of the benefits of 

commissioning has been growing, but the topic is still relatively new. The only relevant 

projects identified in the literature to date involve a total of 37 buildings, of which ten are 

in Texas, 13 are in California, 13 in Oregon and one is in Colorado. 

In retrocommissioned buildings, savings generally decreased with time, but there is wide 

variation from building to building. For the buildings where savings persistence was 

quantified, savings persistence at the time of the study (3 to 8 years after commissioning) 

ranged from about 50 % to 100 % in all but one or two buildings. Average savings at the 

time of the study were about 3⁄4 of the original savings, with the most dramatic savings 

take-backs were caused by undetected mechanical or control component failures. 

In the 10 new buildings studied, over half of the 56 commissioning fixes persisted. 

Hardware fixes, such as moving a sensor or adding a valve, and control algorithm

changes that were reprogrammed generally persisted. Control strategies that could easily 

be changed, such as occupancy schedules, reset schedules, and chiller staging tended not 

to persist. Persistence was also related to operator training. 

 

As is evident, the number of buildings studied here represents a very small portion of 

commercial buildings that have undergone commissioning or retrocommissioning. More 

research is needed to: 

• Develop a uniform methodology for determining commissioning persistence. 

• Determine the persistence of savings from a broader sample of buildings. 

• Develop simple tools for tracking performance of commissioning measures. 

• Develop practical methods for owners and operators to better maintain 

commissioning savings. 

 

Chapter 3: Review of Automated Commissioning Tools for Buildings 

This chapter reviews state of the art automated and semi-automated commissioning tools. 

Automated commissioning is viewed as a means to speed up the commissioning process 

and reduce dependence on scarce and relatively expensive skilled practitioners. A number 

of automated state of the art tools have been developed and tested at research institutions 

and universities with funding from utilities, industry, and government agencies. 

The tools developed to date can broadly be categorized as tools to evaluate the 

performance of systems or tools that automate other aspects of the commissioning 

process. Three commissioning tools (ENFORMA, PACRAT, and Virtual Mechanic1

) 

currently available on the market and six prototype tools are described in the report. 

These tools, which have been developed for a variety of conventional HVAC systems,

address various aspects of initial commissioning and recommissioning/ongoing 

 

1 Disclaimer: Certain trade names and industry standards are mentioned in the text to 

illustrate market-available products. In no case does such identification imply 

recommendation or endorsement by the National Institute of Standards and Technology, 

nor does it imply that the product is the best available for the purpose. 

 2 
commissioning. In addition, there are a number of enabling tools that are valuable 

resources to facilitate the implementation of the commissioning process and the 

development of new tools. 

It is anticipated that the use of automated or semi-automated commissioning tools, 

combined with advances in diagnostics research, will enable more people to perform

these functions in a more efficient manner. Overall, there is good but limited anecdotal 

evidence showing the value of these automated commissioning tools. However, the tools 

are few and are generally limited to selected HVAC systems. Available and emerging 

tools need to be more robust to increase potential applications. This growing area of 

research is expected to further broaden the market for commissioning. 

Chapter 4: Review of Needs and Challenges in Commissioning Zero Energy 

Buildings 

This chapter presents a literature review of commissioning and operational experience 

with existing buildings that were designed to have low energy consumption. Key 

findings from a number of case studies, including the National Renewable Energy 

Laboratory (NREL) High Performance Buildings, are: 

• Designers/ and contractors are not sufficiently familiar with innovative low energy 

designs. 

• There is a lack of commissioning procedures for low energy buildings, so only the 

conventional features get properly commissioned. 

• Deficiencies in design, construction and commissioning result in operational 

problems; however, many of these problems were only detected by the R&D team

studying the building. 

• Controls are a particular source of problems, in part because mechanical designers 

leave the control system design to the controls contractor, who is typically less 

experienced with innovative systems than the design team. 

R&D can help address these problems by: 

• Developing methods of documenting design intent and performing design reviews 

that are adapted to the specific needs of innovative and low energy buildings. 

• Developing functional test methods that adequately address innovative system

operation and integration issues, capable of: 

A. Comparing expected energy performance to actual energy performance 

during commissioning and diagnosing causes of differences. 

B. Incorporating simulation as a means of enforcing accountability for energy 

performance between design and construction. 

Chapter 5: New Functional Tests for use in Commissioning Zero and Low Energy 

Buildings 

This chapter presents four new functional tests for systems with particular 

applicability/importance to low energy buildings: 

• Radiant slabs for heating. 

• Under floor air distribution plenum pressure. 

• Demand-controlled ventilation. 

• Building pressurization. 

 3 
Conclusion 

The overall conclusion of the report is that commissioning has a key role to play in 

comprehensive quality assurance for the design, construction and operation of buildings. 

Quality assurance tools and procedures are necessary to ensure that the technical potential 

of building systems and components is realized in operation throughout the life of the 

building.

 4 
Overview 

Building Commissioning 2is a quality assurance process for the design construction and 

operation of buildings. Although it is recognized as a valuable means to ensure that 

buildings reach their operating potential, the process is not widely adopted. The principle 

barrier to market penetration is the high cost, or the perception of high cost, of 

commissioning. Reducing the cost through automation is one approach to improving 

cost-effectiveness. Documenting the costs and the benefits, and disseminating that 

information is widely seen as critical to increasing the uptake of commissioning. The 

increased use of innovative, interacting, systems in low or zero energy buildings both 

increases the importance of commissioning these buildings and requires the development 

of commissioning methods and procedures for these systems. 

This report documents the findings of a review of the state of the art in the 

commissioning of low energy buildings. The aims of this effort were to: 1) identify 

existing methodologies for defining the costs and benefits of commissioning, including 

the persistence of these benefits, and 2) to assess current practices for commissioning low 

energy buildings and identify the needs for methods and tools that go beyond what 

conventional commissioning approaches can offer. 

The development of standardized methodologies for cost-benefit of commissioning, the 

evaluation of persistence of savings, and automated tools for commissioning are seen as a 

means to break down existing barriers. This literature review seeks to absorb the lessons 

learned in key studies and to distill the information into a format that can useful in the 

development of a plan for future work. The insight gained from this literature review and 

lessons learned from international applications will be used to develop a work plan for 

the International Energy Agency’s Energy Conservation in Buildings and Community 

Systems’ (IEA ECBCS) Annex 47 and will provide input to the Department of Energy’s 

(DOE) effort to develop a multi-year plan for research and development to overcome the 

barriers to widespread use of commissioning in commercial buildings. 

The report is organized as follows. The first chapter presents a review of cost-benefit 

methodologies and data. It presents the findings of 11 studies and discusses data 

collection strategies including the research-driven model, provider-driven model, and the 

database model. It presents methodologies for determining costs, energy benefits and 

non-energy benefits. Finally, recommendations are made for future work. 

The second chapter addresses the persistence of commissioning measures in new and 

existing buildings. It presents detailed summaries of existing building and new building 

 

2 Commissioning- Clarifying Owner's Project Requirements (OPR) from viewpoints of environment,

energy and facility usage, and auditing and verifying different judgments, actions and documentations in 

the Commissioning Process (CxP) in order to realize a performance of building system requested in the

OPR through the life of the building

Initial Commissioning, Re-commissioning, Retro-commissioning, and On-going Commissioning are 

defined in the Glossary of Terms produced by ECBCS Annex 40. 

 5 
studies. Strategies are discussed for improving persistence in new and existing buildings, 

including: design review, building documentation, operator training, building 

benchmarking, energy use tracking, trend-data analysis and re-commissioning. 

The third chapter presents the state of the art in automated commissioning tools. It

presents the three automated commissioning tools that are available commercially along 

with six prototypes that are at various stages of development. Chapter Three also 

presents related research, which includes the development of a number of tools that 

advance the automated tool development efforts. Finally, an overview of the state of 

market penetration is presented that highlights key barriers and assesses the potential for 

automated tools to facilitate aspects of the commissioning process. 

The final chapter addresses the particular issues that pertain to the commissioning of very 

low energy buildings. A literature review of commissioning and operational experience 

with existing buildings that were designed to have low energy consumption has been 

undertaken. Key findings from a number of case studies are reported and generic 

conclusions with implications for R&D are presented. 

 

 6 
Chapter 1: Review of Commissioning Cost-Benefit 

Methodologies and Data

I. Introduction 

This chapter summarizes findings from a review of 11 commissioning cost-benefit 

studies. The chapter focuses specifically on the methodologies used to determine the 

costs and benefits of commissioning. In order to maintain this focus, only studies that 

make their methodologies explicit have been included. The majority of methodologies 

that were analyzed are research studies of multiple buildings, and only a few are case 

studies of just one or two buildings. A more exhaustive list of studies that include cost 

and benefit data, but not an extensive methodological discussion, can be found in the 

bibliography. 

These 11 studies represent a variety of formats and intentions, which were each created to 

meet the funder’s goal. Among them are research reports, databases of cost-benefit 

information and a glossy, marketing-style brochure. Most of the research reports were 

undertaken to produce data to support utility and research programs and to help owners 

and commissioning providers gather the financial justification needed to implement 

commissioning (Cx) or retrocommissioning (RCx). 

There is a significant difference in methodological framework between studies 

implemented as “one-time” or “snapshot” analyses, and those set up to continually collect 

and incorporate new data. It is probably true that any methodology can be implemented 

on a continuous basis if its funding is also continuous. However, data collection 

methodologies that facilitate data entry by allowing respondents to easily enter their own 

data and use an automated or semi-automated analysis tool are better positioned for 

ongoing analysis. 

Tables 1.1 and 1.2 provide a more detailed overview of the studies included in this 

chapter. 

Table 1.2 describes the studies, including their format, expected use and audience, and 

any caveats and considerations that might affect how their conclusions are interpreted.

Four of the studies were originally conducted as research projects, funded by government 

agencies and a non-profit corporation.3

 Nine reported their findings in published 

conference papers.4

 Of the two that were never published as conference papers, one is a 

glossy brochure produced for marketing purposes and the other is an article written for 

subscribers to an energy research and information service.5

 

 

3

 Funders were: U.S. Department of Energy, U.S. Environmental Protection Agency, Lawrence Berkeley 

National Laboratory, Northwest Energy Efficiency Alliance. 4

 Conferences were: American Council for an Energy Efficient Economy Summer Study, National

Conference on Building Commissioning. 5

 Energy information service is ESource. 

 7 
Table 1.2 provides a side-by-side comparison of the studies’ data and findings. They 

represent a wide range of methodological approaches and resulting data on the costs and 

benefits of commissioning. Their data ranges from case studies of one to six buildings to 

more extensive analyses of 16 to 21 buildings to two meta-analyses of data collected and 

analyzed by others, one of 44 buildings and the other of 175 buildings. Among building 

projects studied there is wide range in building size and type and in findings. Cost, 

benefit and simple payback ranges are summarized in Table 1.1: 

Table 1.1. Summary of Costs and Benefits from 11 Studiesi

New construction Existing buildings 

Commissioning costs $2.05/m2

 to $10.76/m2

($0.19/ft2

 to $1.00/ft2) 

$0.86/m2

 to $3.34/m2

($0.08/ft2

 to $0.31/ft2) 

Energy benefits $0.54/m2

 to $6.89/m2 

($0.05/ft2 to $0.64/ft2) 

$1.18/m2

 to $2.80/m2

($0.11/ft2

 to $0.26/ft2) 

Non-energy benefits $1.40/m2

 to $22.60/m2

($0.13/ft2

 to $2.10/ft2) 

$1.18/m2

 to $1.94/m2

($0.11/ft2

 to $0.18/ft2) 

Simple payback 4.8 years to 6.5 years 0.7 years to 3.2 years 

i

Costs and benefits are presented as ranges to demonstrate the variances in the studies examined. Median or average 

values are not presented because underlying methodologies differ widely and such figures would not reflect actual costs 

and benefits experienced by building owners.

 8 
Table 1.2. Description of Cost-Benefit Studies (continued on next page) 

Study/Author Format Use/Audience Caveats and considerationsStum, ECM Cx (1994)

Conference paper Research  Only energy conservation measure (ECM) commissionin

commissioning. 

 Utility program costs are included as a cost of commissi This early study does not address non-energy benefits (N

Piette, Energy Edge Cx(1995)

Technical report This was one of the first studies 

to show savings concretely, and audience is program planners, technology developers. 

 The study is focused on commissioning of ECMs in new 

unrelated deficiencies were reported. 

 Some of the data collection and analysis were associated 

Haasl, 5 Building Study(1996)

Conference Paper Funded by the U.S. 

Environmental Protection Agency, Global Change Division, and the U.S. Department of Energy to help formulate energy conservationpolicy and programs. 

 The study was an “Operations and Maintenance (O&M)  No detail provided on the standard energy calculations o NEB analysis was still in progress.  Stated objective was to “demonstrate that energy saving 

realized through improvements in O&M.” 

PECI/DOE, DeficiencyDatabase (1996)

Research report DOE-funded to document 

deficiencies found through Cxand RCx. 

 While technically not a cost-benefit methodology itself, 

value of incorporating a detailed deficiency database int

 Savings data only available for 35 deficiencies.  A deficiency database may be a lower cost version of a c

Cx reports have some detail on measures at a findings lepunchlists. Cx reports do not always have comparable d

energy savings or non-energy savings and a cost accoundatabase leads to an understanding of where the most co

Gregerson, RCx

(1997)

Report for members of ESource 

Audience was ESource members (utilities, ESCOs, Cxproviders, researchers), to quantify a new field ofefficiency opportunity

 Few reports cited measure costs and savings. Savings m

LoanSTAR program (75 % of square footage in the studestimated. 

 The first major summary report on RCx. 

PECI, Brochure (1997) Glossy brochure Audience was owners and Cx

providers, for marketing. 

 Summary metrics by sector, Cx and RCx mixed. Original data not available.

Altweis(2001) 

Conference paper documenting methodology and detailedassumptions 

Paper’s audience was Cx providers, to encourage them tocollect and report such data.Audience for data is owners andprospective customers.

 Very small sample size, suitable for case studies or resea A wide range in savings reported, due to highly varying  Savings calculation methodology will vary from Cx Age

calculation provided (although the methodology is conce

 No discussion of costs.

 
Study/Author Format Use/Audience Caveats and considerations

Mills, Meta-analysis (2004) 

Excel database, Research report 

Statistical analysis for US Dept. of Energy

 Largest Cx cost-benefit study to date. Focused on obtain

high-level view of Cx metrics. 

 Relied on availability, quality, and comparability of diff Majority of building information comes from a few sour Merits of Cx should be assessed based on the cost-effect

necessarily only on what was implemented.

 May inappropriately attribute or not attribute costs to the 

conventions are not always followed. 

 May underestimate benefits because energy savings from 

reports, NEBs are not usually expressed in monetary termincreased net operating income (NOI) are rarely determistudied, measured savings exceeded predicted savings. 

 Time consuming to gather project information from seco

opposed to having the cost-benefit data entered by the p

SBW, Northwest Cx & RCx (2004) 

Research report Utility program evaluation  Cost calculations include many costs associated with Cx

studies. 

 Non-energy benefits calculations based on opinion of tea

and/or perceived value). 

CaliforniaCommissioning Collaborative’s Cxdatabase.com(2004) 

SQL database, exportable to Excel One-page “datasheet” on each project Conference paper describing database 

Researchers – data that supports Cx research and utilityincentive programs Owners – defining the value ofCx to their business through data and case studiesProviders – third party source from which to give owners information. Help raise the bar for Cx documentation of results 

 Data is stored as-entered by respondents – no analysis pe

researchers

 Datasheet is a one-page summary form automatically po

respondents. 

 Little population of the database as of February, 2006.  Database in beta development level.  While the original vision for this data included creation 

created.

 
II. Data Collection Strategies 

By far the most common data collection method is the Researcher-Driven Model, in 

which a researcher was tasked with collecting and analyzing data. In more than 90 % of 

the studies a researcher was wholly or partially responsible for collecting documentation 

and data produced by others. In the handful that differed from this model, data collection 

was usually done “in house,” because the researcher also served as the commissioning

provider on the projects that were studied. In two cases, however, data collection was 

accomplished through use of a database allowing providers and owners to submit data 

independent of the researcher. A comparison of the data collection methodologies used 

in the different studies reviewed here is presented in Table 1.3. 

The Researcher-Driven Model 

In nine of the eleven studies, the data collection strategy was driven by a researcher who 

collected commissioning project information and produced a cost-benefit report. In eight 

of those nine, the researchers relied heavily on project documentation, primarily the 

commissioning provider’s Final Report. Other documentation consulted included 

construction documents (for new buildings), issue logs and change orders. In more than 

half of the studies, other types of information were used to supplement written 

documentation. They include telephone surveys with key team members (two studies) 

and onsite inspection and monitoring (three studies). 

When telephone surveys or interviews were employed, they were often used to gather 

data on non-energy benefits (NEBs). This is logical, given that NEBs are hardest to 

measure using commissioning or building documentation because they depend most on 

the experiences of the people who manage and occupy the building. In fact, there are two 

studies in which researchers were only interested in NEBs and in which the only source 

of data were telephone surveys and detailed interviews – no project documentation was 

collected (Haasl 1996; Bicknell 2004). 

Among the nine studies that employed the researcher-driven model, there is much 

variation in the amount of data studied and level of detail collected, the logistics of 

obtaining documentation, and in supplemental types of data and the strategies for 

collecting it. 

• Quantity of data varies from case studies of a single project to mid-range studies of 

five, six, 21 and 44 projects to two large studies of 175 projects each. 

• Level of detail ranges from whole-project level metrics to metrics for individual 

issues. 

• Logistics of obtaining documentation includes submission by a utility that collected

all the documentation and turned it over to the researcher, submission by owners and 

providers directly to the researcher, and the gathering of documentation by the 

researchers from commissioning providers and other researchers. 

• Supplemental data includes telephone surveys and onsite inspections.

 11 
In the researcher-driven model, data collection almost always takes place after the 

commissioning projects are complete and documentation finished. As a result, the effort 

required and the data quality depends almost entirely on the diligence of the parties 

responsible for producing the documentation (usually the commissioning provider, 

general contractor or testing and balancing agent). Time is also an issue. The closer the 

study is to project close-out, the more likely it is that project documentation will be 

available and in good condition, and that the important parties will be able to answer any 

questions.

The Provider-Driven Model 

In two studies, the researcher and commissioning provider were one and the same. As a 

result, their studies were able to utilize very detailed data collected throughout the 

commissioning project. However, only a few projects were included, leading to these 

studies’ designation as case studies rather than statistically significant research studies

(Haasl 1996; Altweis 2001). 

The Database Model 

In two studies, researchers created interactive databases to collect commissioning project 

data. In one, the database was created through a collaborative effort in which multiple 

researchers and commissioning providers helped define required and minimum inputs 

(Cxdatabase.com). The database itself was created as an online application, meaning it 

was accessible on the Internet. Thus once it was released, commissioning providers could 

use it to enter information about their projects in real-time. In the other, a database of 

categories deficiencies was developed (PECI/DOE 1996). 

A significant advantage to the database model of data collection is the ability of the 

researcher’s needs to influence the commissioning provider’s data retention efforts. 

Because providers know up-front what data the researcher wants, it can be supplied 

immediately while the documents are still available and the project is fresh in the 

provider’s mind. On the negative side, a database alone is incapable of performing 

analysis, and this model requires funding for several things: design and programming of 

the database, a researcher to analyze the data or work with programmers to build analysis 

functionality into the database, marketing of the database to the provider audience, 

ongoing database maintenance and support, and perhaps even funding to compensate 

providers for entering data. 

Explanation of Estimated Effort 

Table 1.4 includes a column for “Estimated effort need to obtain and enter data.” The 

amount of time and difficulty required to both collect and submit project data is estimated 

as either low, moderate or high. These rankings are not independently defined. Rather, 

they reflect the authors’ estimate of the relative effort required to gather data according to 

the study’s methodology, as compared to the other studies in this report. Thus a study 

with a “high” effort ranking was judged to employ a more time- and effort-intensive 

collection methodology than those deemed “low” or “moderate.”

 12 
Table 1.3. Cost and Benefit Data (continued on next 2 pages)i

Study/ Year # of

buil

dings

T

otal and 

median bldg. size

Costs Energ

y benefits Non-energy benefits

(NEBs) 

Cx RCx Cx RCx Cx RCx 

Stum, ECM Cx (1994) 

6 218 722 ft2

 20 320 m2

$3 060 overall $0.041/ ECM $0.08/ft2

$0.86/m2

Simple

37 412 kWh/y

5.3 % of orig. ECM Unrealized:  7.9 % Moderate 

N/A 

Piette, Energ

y Edge Cx

(1995)

16 849 800 ft2

78949 m227 000 ft2(median) 2 508 m2 (median) 

$0.19/ft2

$2.05/m2

Simple

N/A 9.48

kWh/ft2·y 102.04 kWh/m2

$0.64/ft2·y $6.89/m2y 

Complex 

N/A not quantified N/A 

Haasl, 5 Building Stud

y

(1996)

5 837 000 ft2

77 760 m2

122 000 ft2

(median) 1 313 197 m2 (median) 

N/A $0.11/ft2 

$1.18 m2

Simple

N/A $0.11/ft2

$1.18 m2

Simple

N/A

Not

quantified 

 
Study/ Year # of

buildings

Total and median bldg. size

Costs Energy benefits Non-energy benefits

(NEBs) 

Cx RCx Cx RCx Cx RCx 

PECI/DOE, DeficiencyDatabase (1996)

16 Cx 28 RCx (44 total) 

3 960 000 ft2

367 896 m2

67 000 ft2

6 224 m2

median 

N/A N/A 83 % of all 

deficienciesrelated to energy

92 % ofoperational deficienciesimpact energy.Avg savings/ deficiency = $892/yModerate 

51 % of all deficienciesrelated to reliability and maintenance

25 % ofdeficiencierelated to comfort 

Gregerson, RCx

(1997)

44 8.84 million N/A Approx $20

000 $0.19/ft2

$2.05/m2

Simple

N/A Avg $98 000 

Median: $41 00019.2 % avg savings $0.49/ft2

$5.27/m2

Moderate 

N/A Not assess

PECI, Brochure(1997)

75 Cx and RCx not separated 

Not available Median: 

$0.15/ft2 

$1.61/m2

Not standardized metric Moderate 

Improved ( % of buildings)Thermal comfort: 42 % System function: 44 % Indoor air quality: 23 % O&M: 42 % Simple

Altweis(2001) 

1 14 350 ft2

1 333 m2

not reported 

N/A up to

$0.13/ft2·y 

$1.40/m2

N/A $0.17 -

$2.10/ft2·y 

$1.83 -$22.60/m2y

Moderate 

N/A 

Heinemeier, Schools Cx(2004) 

1 N/A:

methodologybut no results. 

N/A N/A N/A N/A N/A N/A 

i Dollar amounts have not been normalized to a common year. Methodological complexity listed in bold. 6 Energy intensive buildings and even most of the efficient buildings had paybacks of less than two years. 

 
Study/ Year # of

buildings

Total and median bldg. size

Costs Energy benefits Non-energy benefits

(NEBs) 

Cx RCx Cx RCx Cx RCx 

Mills, Meta-analysis(2004)

175 projects RCx: 106 Cx: 69 

30 400 000 ft2

2 824 252 m2

total 69 500 ft26 457 m2

(median Cx) 151 000 ft2

14 028 m2

(median RCx) 

$74 267 $1.00/ft2

$10.76/m2

[0.6 %constr cost]

Moderate

$33 696 $0.27/ft2

$2.90 m2

Moderate

$2533/y$0.05/ft2·y 

$0.54/m2·y 

Moderate

$44 629/y$0.26/ft2·y 

$2.80m2·y 

Moderate

$51 000/y$1.24/ft2·y 

$13.35m2·y 

Moderate

$17 000/y$0.18/ft2·y 

$1.94 m2·y

Modera

SBW, Northwest Cx & RCx (2004) 

21 2.2 million ft2

204 386 m2

$71 7919

$0.85/ft2

$9.15/m2

Complex 

$22 053 $0.31/ft2

$3.34/m2

Complex 

$9 856/y$0.09/ft2·y

$.97/m2·y

Complex 

$13 678/ year $0.14/ft2·y

$1.51m2·y

Complex 

$13 609 (onetime) $0.13/ft2$1.40/m2· Complex 

$10 534 (otime) $0.11/ft2

$1.18/m2

Complex 

CaliforniaCommissioning Collaborative’s Cxdatabase.com

(2004) 

Two surveys completed, five in progress. Not assessed at this time due to lack of data and funding. 

7 Standard energy prices corrected for inflation. Data normalized to $2003. 8 Standard energy prices corrected for inflation Data normalized to $2003. 9 Costs for this study include only Cx provider fees – although payback information includes additional costs, for example, costs to othe

 
Table 1.4. Comparison of data collection methodology (continued on next page) 

Study/Author Level of detail Data sources 

(paid?) 

Collection process Timing of data

collection

Data storag

Stum, ECM Cx (1994)

Only looked at ECMs meriting greaterresources (i.e. variablefrequency drives, economizers) 

Inspection reports, onsite inspection of ECMs in small comm.and retail 

Reports provided byutility, onsite work done by authors. 

Concurrent with and immediately following Cx activities 

N/A 

Piette, Energy Edge Cx (1995)

Very detailed data collection on building characteristics todevelop models, 

Commissioning report, on-site data collection. 

Extensive evaluation project. 

Within one or two years afterconstruction. 

Unix-based d

Haasl, 5 Building Study(1996)

Data required for standard calculations and simulations. 

Provider collected. Data collected through 

RCx process, including two weeks ofmonitored data on keysystems.

Collected during RCx process. 

Not described

PECI/DOE, DeficiencyDatabase (1996)

Findings level detail. Half of data directly

entered by Cx provider. Half byresearcher (paid). 

Review of final commissioning reports and issues logs. 

Retrospective. Database (Ex

Gregerson, RCx (1997) No detail other than 

metrics on a projectlevel. No measurelevel detail. 

Four Cx providers. 70 % from TAMU and 25 % from PECI 

Building characteristics, EUI, and cost and energysavings figuresrequested from Cx provider by researcher. 

Retrospective from final Cx reports. 

Not specified

PECI, Brochure (1997) High level metrics Cx provider Phone interviews Retrospective Not specified 

 
Study/Author Level of detail Data sources Collection process Timing of data 

collection 

Data storag

Altweis(2001) 

Case study level data. Extensive detailed information required from project. Not difficult for the commissioning agent to obtain. 

Commissioning agent collected informationand conducted benefits analysis. 

Commissioning provider review of notes and project documents. 

Throughout the project. 

Not specified

Heinemeier, SchoolsCx (2004) 

Report showed a great deal of detail, but the intent is to definemetrics that are easily gathered, from review of constructiondocuments. 

Complete constructiondocuments. 

Researcher obtained acopy of and reviewed all constructiondocuments. 

After the project was complete. 

Not specified

Mills, Meta-analysis (2004) 

Based on the documentation available. Where little available, at minimum,project-level info was entered. 

A few Cx providers and researchers entered many projects (paid). Smaller number of projects from unpaid Cx providers. 

Review of past studies and final Cx reports/issues logs. 

Projects completed between 1993 and 2004 Retrospective from final Cx reports and previous studies. 

Excel spreads

SBW, Northwest Cx & RCx (2004) 

Identified all issues/findings, selected onlysignificant and resolved issues

Extensive project documentation and surveys (both unpaid). 

Project materialssubmitted by owner and telephone surveyswith team. 

While projects underway and within 1year after close-out (early 2003) 

Database (no software iden

CaliforniaCommissioning Collaborative’s Cxdatabase.com (2004) 

Three findingsrequired, canaccommodateunlimited number 

Cx provider or owner. (unpaid, but funding for entering data desired, requirement was written into scope of some projects) 

Respondent gathers data and enters into online forms. 

Intended to be completed during project or immediatelyafter completion. Canbe completed at any time, if data is available. 

Custom-built database – prseveral monthof approx. $2(www.Cxdata

 
III. Methodologies for Determining Costs, Energy Benefits and 

Non-Energy Benefits 

Commissioning Costs 

There is no widely used methodology for determining commissioning costs. To assist in 

the evaluation process, this report distinguishes three levels of complexity in cost 

methodologies: simple, moderate and complex. 

Table 1.5 summarizes their differences in terms of which costs are included and if the 

costs are validated. 

Table 1.5. Comparison of cost methodologies 

Cx Provider’s fee Resolution costs 

(RCx) 

Costs to other 

parties Validation method 

Simple X X

Moderate X X X

Complex X X X X

Of the 11 studies examined, all include the commissioning provider’s fee as a cost of 

commissioning. Some include additional costs, for example, costs to other parties, 

although each study defines these costs differently. Only one study, with a complex 

methodology, makes an attempt to validate cost data by checking the respondent's data 

for consistency and to make sure cost figures fell within what researchers defined as a 

"reasonable range." (SBW Consulting 2004) 

In general, the average cost of commissioning per square foot increases as the study’s 

cost methodology increases in complexity. As Table 1.6 shows, in existing buildings the 

cost of commissioning steps upward as the methodology becomes more complex. In the 

case of new buildings, the cost of commissioning trends higher, with complex 

methodologies returning an average cost per square foot (square meter) slightly lower 

than moderate methodologies. Although not conclusive, it seems likely that the reported 

cost increases because complex methodologies account for costs incurred by several 

parties, whereas simple methodologies usually only account for the commissioning 

provider’s fee. 

 18
Table 1.6. Average Cost of Commissioning by Methodological Complexity

New Buildings Existing Buildings 

Methodology # of bldgs Average Cost # of bldgs Average cost 

Complex 13 

$0.85/ft2

$9.15/m2 8 

$0.31ft2

$3.34m2

Moderate 69 

$1.00/ft2 

$10.76/m2 106 

$0.27ft2

$2.91m2

Simple 16 

$0.19/ft2

$2.05/m2 50 

$0.18ft2

$1.90m2

Below is a more detailed discussion of cost methodology types, with examples. 

Simple. A simple methodology uses only one or two cost categories to arrive at the 

overall cost of commissioning. Usually these cost figures are relatively easy to obtain. 

Examples include the commissioning provider’s fee and the cost to resolve an issue. In 

simple methodologies, these cost figures are self-reported and the study makes little or no 

attempt to validate the data. 

An example of a simple methodology can be found in two early studies: Piette (1995) and 

Stum (1994). Piette calculated the cost of commissioning by taking a percentage of the 

overall energy efficiency measure cost. Stum defined the cost of commissioning as the 

self-reported commissioning provider’s plus the administrative costs of the utility 

commissioning program that funded the projects. 

Moderate. A moderately complex methodology uses more than two cost categories to 

arrive at the overall cost of commissioning. For example, cost categories could include 

incremental costs to all parties, travel costs, and negative impacts like increased change

orders. Moderately complex methodologies include a broader array of costs in the cost of 

commissioning than a simple methodology, although the study stops short of applying a 

validation process to the data. The methodology may include differences in cost 

accounting between Cx and RCx. 

An example of a moderate methodology can be found in Mills et al. (2004) and in 

Cxdatabase.com (2004). Mills et al.’s cost definition includes several figures: the 

provider fee, the coordination costs incurred by other parties and on RCx projects, the 

resolution costs. Cxdatabase.com’s cost definition includes the provider fee, incremental 

costs incurred by other parties, the cost of O&M staff participation (if specified by the

owner) and on RCx projects, the resolution costs. Neither Mills et al. nor Cxdatabase.com

makes any attempt to verify the cost figures reported in project documentation or by 

respondents. 

Complex. Like a moderate methodology, a complex methodology differentiates several 

categories of commissioning costs. However, studies employing complex methodologies 

do attempt to validate cost figures. 

 19
An example of a complex methodology is found in SBW Consulting (2004). Cost 

includes the provider fee, incremental fee increases for other parties, travel expenses and 

resolution costs. Researchers used a telephone survey to ask key commissioning team 

members 1) if they increased their bid for the project to account for commissioning 

activities and 2) if there were any significant non-labor costs associated with 

commissioning. Respondents were then asked to attach a dollar amount to each. If the 

respondent was unable to provide a dollar figure, researchers asked them to estimate the 

additional labor hours and provide a labor rate, from which researchers calculated the 

incremental cost to that respondent. As a quality assurance measure, researchers also 

evaluated whether the data supplied by respondents “were consistent and fell within 

reasonable ranges” (SBW Consulting 2004). 

Energy Savings 

There are a variety of methods for determining energy savings from commissioning. This 

study evaluates energy benefit methodologies, like cost methodologies, according to their 

level of complexity. 

Table 1.7. Comparison of energy savings methodologies 

Issues ID 

and/or 

baseline 

comparison 10

Energy

calculations

Energy

modeling 

Normalization 

of energy

data

Attention to

measure 

interactions

Validation

method 

Simple X

Moderate X / / / /

Complex X X X X X X

Key: X = always present; / = sometimes, but not always present 

Of the 11 studies examined, all used either an issues identification or baseline comparison 

method to determine energy savings. Moderate methodologies employed some form of 

energy calculations, modeling, or data normalization. Only complex methodologies were 

attentive to measure interactions and data validation. 

Below is a more detailed discussion of cost methodology types, with examples. 

Simple. In existing buildings, simple methodologies compare before and after energy 

consumption without normalization of data. They may also obtain information directly 

 

10 Two methods for determining energy (and non-energy) savings are issue identification and baseline 

comparison. In issue identification energy savings are determined first at the issue level and then added to

arrive at the total savings for the project. The baseline comparison method looks only at whole-building

energy benefits. The researcher establishes the building’s “baseline” energy use and then compares it to

energy use after commissioning. This method can be a more straightforward process in existing building 

projects, where there is a “before” snapshot. In new construction it is more difficult because the “baseline” 

is hypothetical and must be simulated.

 20
from the building owner or manager regarding energy savings or comparisons of 

performance. 

Examples of simple energy benefit methodologies are found in Cxdatabase.com (2004) 

and Heinemeier (2004). Cxdatabase.com asked survey respondents to provide energy 

savings numbers for each reported finding. Respondents were asked to also provide the 

calculations they used to arrive at the figures, but this information was not required. No 

standardized process for calculating energy savings was created. In Heinemeier’s 

methodology, energy use per square foot of commissioned buildings was compared to 

those that were not commissioned. Building pairs were of similar size and type, and 

monthly utility bills were used to gather energy use data. Commissioned building energy 

use was also compared against standardized benchmarks. 

Moderate. Moderately complex methodologies use project documentation to identify 

significant commissioning findings/issues that have been resolved, and then use 

engineering calculations or parametric modeling to determine the energy benefit. A 

validation process using measured data may be, but is not necessarily, applied. 

Moderately complex methodologies may also apply normalization techniques to before 

and after energy consumption. 

An example of a moderately complex energy benefit methodology is found in the SBW 

Consulting (2004). Researchers used a three-step process, shown in Figure 1.1, to identify 

issues that resulted in a “stream” of energy and/or non-energy benefits. First they used 

Figure 1.1. Sample issue identification methodology (SBW Consulting, 2004) 

 21
project documentation to identify all issues. Then they determined which issues were 

“significant” relative to their affect on total building area or occupants, resolution cost 

and/or long term impact. (SBW Consulting 2004) Of significant issues, they determined 

through documentation and/or telephone surveys which issues had been or would likely 

be resolved. Energy and non-energy benefits were only calculated for issues deemed 

significant and resolved.

Complex. Studies utilizing complex methodologies employ detailed engineering 

calculations or models to estimate energy savings. Examples range from detailed building 

simulations that require extensive information about building characteristics to very 

detailed engineering calculations based on measured data. Complex methodologies for 

new construction commissioning benefits address nuances such as the range of 

assumptions that go into the hypothetical baseline (i.e. what is assumed to have occurred 

without commissioning). More complex methods also address the interaction between 

commissioning measures, and the interaction with related activities like energy retrofits. 

Results can be reported per measure, or for a whole building (which is not simply the 

sum of individual measures). 

An example of a complex energy benefit methodology is found in Altweis (2001). This 

study used engineering calculations to estimate energy use both with and without 

identified findings/issues. Researchers developed both a “most likely” and a “least cost

solution” scenario, depending on assumptions about what would have occurred in the 

absence of commissioning. 

Non-Energy Benefits (NEBS) 

In the assessment of non-energy benefits there is a great diversity of methodologies in 

use. Here again, in Table 1.8, non-energy benefit methodologies are classified according 

to their complexity. 

Table 1.8. Comparison of non-energy benefit methodologies 

Monetary value NOT 

assigned

Monetary value

assigned

Monetary value

validated

Simple X

Moderate X

Complex X X

Of the 11 studies examined, about half (five) do not assign a monetary value to NEBs. 

Those that do use methodologies ranging from simple processes that do not employ 

standard calculations or checks on respondents’ information to a highly complex system

in which NEB dollar values are calculated several different ways and the most 

conservative number selected. Here too, only the most complex methodologies attempt to 

validate the data. 

 22
Contrary to the direct relationship between methodological complexity and 

commissioning costs, with NEBs there appears to be an inverse relationship: the more 

complex the methodology the lower the monetary benefit reported. This holds true for 

both new and existing building commissioning. 

It is apparent that methodology significantly impacts reported non-energy benefits. The 

study employing avoided cost calculations (a moderate methodology) returned higher 

savings than the study that determined the owner’s perceived value of the benefit (a 

complex methodology). This is due to a fundamental difference between these two 

methodologies. Although both methods are hypothetical, avoided cost calculations 

estimate the full cost that would have been incurred, had the benefit not been received. 

Whereas the owner’s perceived value is the amount the owner is willing to pay for the 

benefit – often less than the avoided cost. Further study in this area is clearly needed to 

determine how the non-energy benefit valuation method relates to the goals of the costbenefit methodology. 

Table 1.9 Non-Energy Benefits of Commissioning by Methodological Complexity

i

New Buildings Existing Buildings

Methodology

# of 

buildings Average NEB Savings # of buildings 

Average NEB 

Savings 

Complex 13 $0.13/ft2

$1.40/m2 8 $0.11/ft2

$1.18/m2 

Moderate 23 $0.17/ft2

 to $6.96/ft2 

$1.83/m2 to $74.92/m2 10 $0.10/ft2

 to $0.45/ft2

$1.08/m2

 to $4.84/m2

Simple no data no data 

i Moderate data is presented as a range because a validation method was not employed.

Below is a more detailed discussion of non-energy benefit methodology types, with 

examples. 

Simple. An example of a simple methodology for assessing non-energy benefits is found 

in Cxdatabase.com (2004). Here, respondents are asked to identify which benefits they 

received and have the option, but not the requirement, to supply a dollar value for the 

benefit. No standardized calculations are employed, and there is no process for evaluating 

the dollar values supplied by respondents.

Moderate. An example of a moderately complex methodology for calculating NEBs is 

found in Mills et al. (2004). Here, the researcher arrived at the NEB dollar value by 

adding the first-cost dollar value of non-energy savings and the ongoing labor cost 

savings, estimated as labor hours saved. Other NEBs were accounted for using a Yes/No 

checklist with an estimated dollar value supplied optionally. 

Complex. An example of a complex methodology for assessing NEBs is found in SBW 

Consulting (2004), see Figure 1.2. Researchers developed three different ways to assign a 

dollar value to a “stream” of benefits flowing from a specific finding/issue, and then used 

the most conservative (lowest dollar value) estimate. All three calculations were based on 

 23
the responses of key commissioning team members given in telephone surveys. (See 

Table 1.9 for additional details).

1. Willingness to pay (WTP)

Survey question: 

“If all the non-energy benefits (and negative effects) that we talked about were taken away, what do

you think would be the maximum amount you would be willing to pay to get back those benefits, on 

an annual or monthly basis?”

2. Sum of individual computed benefits 

Respondents asked to compare the commissioning provider’s fee to the specific commissioning 

benefit. All benefits are then summed for a total NEB value. An example survey question:

“Would you say that compared to your annualized commissioning costs, the contractor call-backs 

are...about 10 % more valuable, about 1 to 1.5 times more valuable, twice as valuable, more than

twice as valuable?” Or, “Don’t know/refused.” 

3. Overall net value 

Respondents were asked to identify significant impacts from a given list (e.g., reducing operational 

deficiencies). Their responses were weighted, to give the opinions of providers and facility staff more 

importance than contractors and designers. The gross dollar value of each impact was then multiplied

by the importance factor. 

Figure 1.2. Sample complex NEB methodology (SBW Consulting, 2004) 

 24
Table 1.10. Overview of cost-benefit evaluation methodologies (continued on next 3 pages) 

Study/Author Costs Energy Benefits Non-energy Benefits (NEBs) Stum, ECM Cx (1994)

Simple Cost of Cx 

provider services + program administration costs 

Moderate Engineering calculations

and computer simulations from original ECM savings predictions 

N/A 

Piette, Energy Edge Cx(1995)

Simple Percent of overall 

energy efficiencymeasure cost, costsby energyefficiency measure are used whenavailable. 

Complex Used deficiency

identification and postconstruction utility data to tune as-builtsimulation models. Deficiencies were“removed” from the models to determine savings. Deficiencies categorized as Directly,Indirectly, and Un-relatedto the ECM, and Static orDynamic. 

Simple Categorized as control, IEA, 

equipment life, O&M. No value assigned.

Haasl, 5 Building Study(1996)

N/A N/A Simple Categorize deficiencies

into deficiency type(maintenance, documentation, training, operations, installation, design); HVAC subsystem; and affectedcomponent. Additional categorization for controls related findings. 

Simple All deficiencies were 

categorized, including nonenergy related deficiencies.In total (Cx and RCx), 51 % of all deficiencies related to reliability and maintenance (25 % significantly related). In total (Cx and RCx), 25 % of all deficiencies related to comfort (13 % significantlyrelated). 
Study/Author Costs Energy Benefits Non-energy Benefits (NEBs) 

PECI/DOE, DeficiencyDatabase (1996)

Moderate Categorized as

Assessment or Implementation.Includedconsultant, contractor, andbuilding staff time, as well asparts and leasecosts for monitoring equipment. Didnot include “research”related costs. 

Moderate “Potential” energy

savings reported. A combination of standardengineering calculationsand DOE2 building simulations, with shortterm diagnostic monitored data used to inform the calculations or model. 

Moderate Only includes extended

equipment life, which is themost easily quantified effect.Categorized as extendedequipment life through reduced hours of operationand through reduced short cycling. Calculated based onassumptions of reducedhours, reduced lifetime through short cycling, and nominal life. 

Gregerson, RCx

(1997) 

Simple Costs reported 

or estimated byeach Cx provider. Costs includeCx fee, monitoring costs, and the cost of implementing measures exceptfor in-house facility stafftime duringnormal working hours. 

Moderate Project documentation

may have utilized engineering calculations, models, or pre- and postconsumption measurement to quantify savings. Report notes that rigor with which energy savings were calculated variessignificantly. 

N/A 

PECI, Brochure(1997)

Simple Cost range and

median cost. 

Simple Savings range by

building type. Conductedphone interviews. 

Moderate Identified NEB

quantitatively for manydifferent categories 
Study/Author Costs Energ

y Benefits Non-energy Benefits (NEBs) 

Altweis(2001) 

N/A

Complex Used engineering 

calculations to estimate energy used with and without identified deficiencies. Provided Most Likely and Least Cost Solution scenarios,depending on assumptions for what would have occurred absent commissioning. 

Moderate Used simple calculations and

extensive assumptions toestimate impacts to factors such as lost productivity, lostsales due to late building completion and equipment replacement. Provided Most Likely and Least Cost Solution scenarios,depending on assumptions for what would have occurred absentcommissioning. 

Heinemeier, Schools Cx(2004) 

N/A. Simple Comparison of monthly

utility bills (electricityand gas (kBtu) per squarefoot), between commissioned and uncommissionedbuildings (well 

matched 

pair or large sample size recommended), also comparing com

missioned 

buildings withbenchmarks (e.g., CBECS). 

Moderate Comparison of well-defined

metrics collected during construction and operationphases, betweencommissioned and uncommissioned buildings(well matched pair or large sample size recommended). 
Study/Author Costs Energy Benefits Non-energy Benefits (NEBs) 

Mills, Meta-analysis(2004) 

Moderate Cx and RCx: 

includes Cx provider fee, Cxcoordination costs of other parties Cx: Does not include resolution cost for “qualityassurance”findings or cost to fix designflaws RCx: Includes resolution cost 

Moderate Project documentation

may have utilized engineering calculations, models, or pre- and postconsumption measurement to quantify savings. 58 % of RCx and 28 % of Cx projects verified measures to be implemented. 

Moderate First cost non-energy savings

($), and ongoing labor cost savings (type, labor hours saved), and includes a list of other NEBs (Y/N, $) 

SBW, Northwest Cx & RCx (2004) 

Complex Includes 

incremental feeincreases, travel expenses, and resolution costs to each party, asreported byrespondents. 

Moderate Used project 

documentation to identified significant andresolved issues, then used standard engineering calculations or parametricmodeling to get savings. 

Complex Dollar value estimated three 

different ways based on telephone survey data with most conservative figure used. 

CaliforniaCommissioning Collaborative’s Cxdatabase.com(2004) 

Moderate Includes minor

capitalimprovements as a cost ofRCx. 12

Includes incrementalcosts to other parties 

Simple Respondent provides info

for energy-savings calculations for eachfinding, not required to perform calculation. Persistence and avoidedcost info optional. Nostandardized calculations for energy savings. 

Simple Respondents asked to

identify which benefits theyreceived, and given the option of entering a dollar value for the estimated avoided cost. Nostandardized calculations foravoided costs. 

11 The fast payback times for Cx measures are most likely shorter than the period of erosion of savings.12 Allows owner to specify whether O&M staff participation is a cost or a benefit. Does not include resolution costs for “quality assurance” findings a
IV. Recommendations, Decision Points, and Next Steps 

Table 1.10 displays the 11 diverse studies reviewed in this report. The studies represent a 

large range of data collection, costs, energy benefit and non-energy benefit 

methodologies. Although this makes generalizations difficult, their collective efforts 

point to several recommendations moving forward, and several decision points to which 

any new study must attend. 

Recommendations 

1. Building commissioning data should be greatly diversified. In the majority of these 

studies, building information comes from only a few sources, like a handful of 

commissioning providers or a large university research department. It is thus unclear how 

well the findings of these studies will apply to the worldwide commissioning industry. 

Moving forward, an attempt should be made to gather building data from a much broader 

base. To date, it has been difficult to collect data from diverse projects because owners 

do not tend to ask for (or pay for) this kind of data on their own projects, and 

commissioning providers therefore do not gather it. Collecting commissioning data in a 

consistent way requires artificial injection of a research project or program to help 

standardize the way data is gathered and reported by market actors. 

2. A complex cost-benefit methodology may require continuous data collection 

throughout the commissioning project, extensive interviews, or both, to acquire a 

sufficiently detailed reporting of costs. The data required for complex, and sometimes 

even moderate, cost methodologies will be difficult to obtain with a retroactive data 

collection methodology relying solely on documentation. It is nearly impossible to 

determine from documentation costs that are not explicitly defined during the project. For 

example, a study may want to include in its cost calculation the cost to the contractor of 

coordinating with the commissioning provider. If this cost is not defined either during or 

immediately after the commissioning process it will not be included in documentation 

(although it may be obtained through a timely interview). As a result, retroactive studies 

relying mostly on project documentation are often forced to “take what they can get,” a

methodology which does not lead itself to a consistent definition of commissioning costs. 

A study employing a complex cost methodology should facilitate data entry by using a 

collection methodology that allows respondents to easily enter their own data and thus 

helps avoid the need for retrospective information gathering based on project 

documentation. The use of an automated or semi-automated analysis tool that positions 

the effort for ongoing analysis should also be considered. 

3. Data validation is an important aspect of any cost-benefit methodology. In cases 

where data is not verified, the accuracy of the cost-benefit results may be at risk. 

Decision Points in Creating a Cx/RCx Cost-benefit Methodology

Creating an appropriate and feasible commissioning cost-benefit methodology that 

achieves the goals of the project requires careful planning around some key decision 

points. Ultimately, these decisions lead to a methodology that can have a range of levels 

 29 
of effort to collect and analyze data, as is shown in this chapter. The following key 

decision points emerged during this analysis of cost-benefit methodologies. 

General 

• What is the goal of data collection and who is the expected audience? 

• What data and format are appropriate to the study goals and audience? 

• What resources do researchers have available to them? This includes both financial 

resources as well as current and potential data sources. 

• How important is verification of data? (Possible levels: reasonableness check, 

oversight of energy and non-energy benefits calculations, and verification for 

persistence.)

Data Collection 

• Will the study be a one-time event that looks retrospectively at past projects, or will 

data collection and analysis occur continuously with current and future projects?

Costs 

• Should the cost to resolve problems identified by the commissioning provider be 

counted as a cost of commissioning? If these resolutions are major design changes, 

should they be counted as a cost of commissioning?

• Should the commissioning-related costs of designers, contractors, and operating staff 

be counted as costs of commissioning?

• Are tasks performed by a commissioning provider that are out of the scope of 

commissioning counted as a cost of commissioning? For example, designers are 

generally tasked with developing the design intent documents. If the designer does 

not complete these documents, but the commissioning provider must have a complete 

set to functionally test the systems against, then often the commissioning provider 

will complete the task. 

• Are costs treated differently for new construction commissioning and 

retrocommissioning?

Energy Benefits 

• Will the methodology be whole building or measure based?

• Will the methodology require monitored data or rely on calculations, and will 

calculations be validated by monitored data?

• How will measured data be collected (e.g., utility bills, dataloggers, or trends from the 

building automation system)?

• How will it be tracked that identified measures are implemented?

• Will standardized calculations be used, or guidelines for calculations or modeling?

• What standardized documentation must be collected to support modeling or 

calculation? 

• How will persistence of savings be estimated or verified? 

 30 
Non-Energy Benefits 

• Will an attempt be made to quantify the financial consequences of non-energy 

benefits?

• If not, how will non-energy benefits be reported and verified?

• If so, will the financial non-energy benefit be self-reported, or will a verification 

methodology be employed?

Next Steps 

This chapter has described and compared how different studies have tackled creating a 

cost-benefit methodology for commissioning. Going forward, a standard cost-benefit 

methodology for an international audience will be created and populated as a part of the 

new IEA Annex 47. This effort will begin with discussions on how the US Department of 

Energy and Annex 47 members wish to proceed with the decision points listed above. 

These decisions must be made with an understanding of the level of funding and effort 

each country can contribute to gathering data. The first meeting of Annex 47 in Fall 

2005 will begin this international planning effort to move the creation of a cost-benefit 

methodology forward. 

References 

11 Key Studies 

Altwies, Joy and Ian B. D. McIntosh. 2001. “Quantifying the Cost Benefits of 

Commissioning” in Proceedings of the 9th National Conference on Building 

Commissioning. PECI. 

Friedman, Hannah, Tudi Haasl, and Ken Gillespie. 2004. “Creating California’s Online 

Commissioning Case Study Database: Case Studies Go High Tech” ” in Proceedings of

the ACEEE 2004 Summer Study on Energy Efficiency in Buildings, 3-64 – 3-75, 

Washington, DC: ACEEE. 

Gregerson, Joan. 1997. Commissioning Existing Buildings. E Source TU-97-3. Boulder, 

CO: E Source, Inc. 

Haasl, Tudi, K. Stum, and M. Arney. 1996. “Better Buildings through Improved O&M – 

A Five Building Case Study.” in Proceedings of the 5th National Conference on 

Building Commissioning. 

Heinemeier, Kristin, Michael Martin, Balaji Santhanakrishnan, Anita Ledbetter, Jim

Shoop, Wes Harvey, Joseph Martin, and Frank Thomas. 2004. “Commissioning of New 

Schools: A State Funded Study of Costs and Benefits” in Proceedings of the ACEEE 

2004 Summer Study on Energy Efficiency in Buildings, 3-88 -3-99, Washington, DC: 

ACEEE. 

 31 
Mills, Evan, Hannah Friedman, Tehesia Powell, Norman Bourassa, David Claridge, Tudi 

Haasl and Mary Ann Piette. 2004. “The Cost-Effectiveness of Commercial-Building 

Commissioning.” LBNL – 56637, December, 2004. 

Piette, Mary Ann, B. Nordman, and S. Greenberg. 1995. “Commissioning of EnergyEfficiency Measures: Costs and Benefits for 16 Buildings.” LBNL – 36448, April. 

Piette, Mary Ann and Bruce Nordman. 1995. “Energy and Cost Savings from

Commissioning Efficiency Measures in 16 Buildings” in Proceedings of the 4th National 

Conference on Building Commissioning. 

Portland Energy Conservation, Inc. (PECI). 1997. “What Can Commissioning Do for 

Your Building?” 

PECI. 1996. “Building Systems Deficiency Database.” Report to the U.S. Environmental 

Protection Agency/U.S. Department of Energy under Assistance Agreement CX822837-

01-0, December, 1996. 

SBW Consulting. 2004. “Cost-Benefit Analysis for the Commissioning in Public 

Buildings Project,” submitted to the Northwest Energy Efficiency Alliance, posted May, 

2004. 

Tso, Bing, Marc Schuldt, and Jun Quan. 2004. “The Costs and Benefits of Achieving 

Silver LEED for Two Seattle Municipal Buildings” in Proceedings of the ACEEE 2004 

Summer Study on Energy Efficiency in Buildings, 3-338 -3-350, Washington, DC: 

ACEEE. 

Tso, Bing, Michael Baker, and Andrzej Pekalski. 2002. “The Costs and Benefits of 

Commissioning Oregon Public Buildings” in: Proceedings of the ACEEE 2002 Summer 

Study on Energy Efficiency in Buildings, 3.373-3.384, Washington, DC: ACEEE. An 

earlier version is also included, from the Proceedings of the 8th National Conference on 

Building Commissioning, 2000. PECI. (Based on SBW Consulting. “Cost-Benefit 

Analysis for the Commissioning in Public Buildings Project”) 

Stum, Karl, Tudi Haasl and Dan Krebs. 1994. “Costs and Savings of ECM 

Commissioning and Inspection – Case Studies from One Utility” in Proceedings of the 

2nd National Conference on Building Commissioning, PECI. 

Additional Cost-Benefit Studies 

Bicknell, Charles and Liza Skumatz. 2004. “Non-Energy Benefits (NEBs) in the 

Commercial Sector: Results from Hundreds of Buildings” in Proceedings of the ACEEE 

2004 Summer Study on Energy Efficiency in Buildings, 4-10 – 4-22, Washington, DC: 

ACEEE. 

Bushell, Christopher, Jack Wolpert, Mandeep Singh, and William Burns. 2002. “Retro 

Commissioning at Chicago Public Schools” in Proceedings of the 10th National 

Conference on Building Commissioning, PECI. 

 32 
Chao, Mark and David B. Goldstein. 1998. “Commissioning, Energy Efficiency, and the 

Asset Value of Commercial Properties: Perspectives from Appraisers and Financial 

Stakes for Owners” in Proceedings of the 6th National Conference on Building 

Commissioning, PECI. 

Claridge, David E., Charles H. Culp, Song Deng, W.D. Turner, J.S. Haberl, and 

Mingsheng Liu. 2000. “Continuous Commissioning of a University Campus” in 

Proceedings of the 8th National Conference on Building Commissioning, PECI. 

Cohan, David and Phil Willems. 2001. “Construction Costs and Commissioning” in 

Proceedings of the 9th National Conference on Building Commissioning, PECI. 

Coleman, James D. 1998. “Three Building Tune-Up Case Studies” in Proceedings of the 

6th National Conference on Building Commissioning, PECI. 

Cox, Robert and April Williams. 2000. “Quantifying Costs & Benefits of

Commissioning” 

in Proceedings of the 8th National Conference on Building Commissioning, PECI. 

Deall, Jerry H., Jack S. Wolpert, Mandeep Singh, and James Kelley. 2002. “Savings Due 

to Commissioning at the King Center, (Performing Arts Building)” in Proceedings of the 

10th National Conference on Building Commissioning, PECI. 

Dorgan, Chad, Robert Cox, and Charles Dorgan. 2002. “The Value of the 

Commissioning Process: Costs and Benefits.” The Austin Papers: Best of the 2002 

International Green Building Conference, Compiled by Environmental Building News, 

pp. 25-30. 

Khan, Aleisha. 2002. “Retrocommissioning of Two Long-Term Care Facilities in 

California. 

Institute for Market Transformation 2002, available online at www.imt.org/papers.

Kumar, Satish and William J. Fisk. 2002. “IEQ and the Impact on Building Occupants” 

in the 

ASHRAE Journal. 

Kumar, Satish and William J. Fisk. 2002. “The Role of Emerging Energy-Efficient 

Technology in Promoting Workplace Productivity and Health: Final Report” Publication 

source unknown. February 13, 2002. 

Martinez, Mark S. 1999. “Energy Efficient Retrofit Commissioning – Tricks of the Trade 

to 

Avoid Claims and Unhappy Customers” in Proceedings of the 7th National Conference 

on Building Commissioning, PECI. 

McHugh, Jonathon, Lisa Heschong, Nehemiah Stone, Abby Vogen, Daryl Mills, and 

Cosimina Panetti. 2002. “Non-Energy Benefits as a Market Transformation Driver” in 

 33 
Proceedings of the ACEEE 2002 Summer Study on Energy Efficiency in Buildings,

6.209-6.220, Washington, DC: ACEEE. 

Milton, Donald K. 2002. “IEQ and the Impact on Employee Sick Leave” in the ASHRAE 

Journal, 2002. 

Nelson, Norman L. 1999. “Avoiding Litigation through Proper Commissioning: My 

Experience as an Expert Witness” in Proceedings of the 7th National Conference on 

Building Commissioning, PECI. 

Nicholls, Andrew, Sean McDonald, and John D. Ryan. 2002. “Estimating the Benefits 

from Building Technologies: Issues, Challenges, and Lessons Learned from Blue Ribbon 

Panel Reviews” in Proceedings of the ACEEE 2002 Summer Study on Energy Efficiency 

in Buildings, 10.221-10.236, Washington, DC: ACEEE. 

Parker, Gretchen and Mark Chao. 1999. “Management and Documentation of Building 

Energy Performance by Real Estate Investment Trusts” in Proceedings of the 7th 

National Conference on Building Commissioning, PECI. 

Parker, Gretchen, Mark Chao, and Ken Gillespie. 2000. “Energy-related Practices and 

Investment Criteria of Corporate Decision-Makers” in Proceedings of the ACEEE 2000 

Summer Study on Energy Efficiency in Buildings. Washington, DC: ACEEE. 

Parks, Jim, Mazin Kellow, Debby Dodds, and Greg Cunningham. 2001. “Getting the 

Most Out of Your Building: SMUD’s Commissioning Program” in Proceedings of the 

ACEEE 2000 Summer Study on Energy Efficiency in Buildings, 4.247 - 4.256, 

Washington, DC: ACEEE; also published as “SMUD’s Retrocommissioning Program: 

Improving With Time” in Proceedings of the 9th National Conference on Building 

Commissioning, PECI. 

Tseng, Paul. 1998. “Building Commissioning: Benefits and Costs.” Heating/Piping/Air 

Conditioning, April, 1998, pp. 51-59. 

Veltri, Anthony. 2002. “Development of an Integrated Commissioning Strategy and Cost 

Model” in Proceedings of the 10th National Conference on Building Commissioning, 

PECI. 

Wei, Guanghua, Mingsheng Liu, Martha J. Hewett, and Mark W. Hancock. 2001. 

“Continuous Commissioning of a Hospital Complex” in Proceedings of the 9th National 

Conference on Building Commissioning, PECI. 

Wilkinson, Ronald J. 2000. “Establishing Commissioning Fees” ASHRAE Journal. 

February 2000, pp. 41-51. 

 34 
Chapter 2: Review of Persistence of Commissioning 

Measures in New and Existing Buildings 

I. Introduction 

In recent years the topic of persistence of benefits has gained more interest both for 

existing building retrocommissioning and new building commissioning. Several studies 

have been performed and published examining both aspects of this topic. This review 

will summarize the key results of these studies. The categories presented are persistence 

of commissioning measures in existing buildings, persistence of commissioning measures 

in new buildings, strategies for improving persistence in new and existing buildings, and 

related reports. This topic is relatively new, and the only relevant projects identified in 

the literature to date involve a total of 37 buildings as noted below: 

• 10 Retro Commissioned Buildings at Texas A&M University – Turner et al. (2001) 

and Claridge et al. (2002, 2004) 

• 8 Retro Commissioned Buildings in Sacramento, California – Bourassa et al. (2004) 

• 8 Retro Commissioned Buildings in Oregon – Peterson (2005) 

• 1 Retro Commissioned Building in Colorado – Selch and Bradford (2005) 

• 10 Commissioned New Buildings – Friedman et al. (2002, 2003a, 2003b) 

Since the total literature identified consists of published papers and reports from only five 

projects directly related to persistence, the summaries presented for each project are 

considerably more detailed than is customary in a literature review. 

II. Persistence of Commissioning Measures in Existing 

Buildings 

10 Buildings at Texas A&M University

A study was performed in 2000 to evaluate the persistence of savings in 10 buildings on a 

university campus three years after the buildings participated in retrocommissioning 

(Turner et al. 2001). The objectives of the study were to determine quantitatively how

much savings degradation occurred and the major causes of any observed degradation. 

The investigation did not focus on the detailed measures implemented in each building

but rather on the degree to which the measures implemented in the retrocommissioning 

process had been maintained, as indicated by examination of energy use data, the 

retrocommissioning reports, and the control settings in place on the main energy 

management control system.

The study was conducted in five major parts. First, buildings were selected to be studied. 

Second, savings calculations were performed based on energy usage data from the 

different periods needed. Third, field examination and commissioning follow-up was 

 35 
conducted on two buildings in which major savings degradation occurred. Fourth, 

operational and controls changes that could have contributed to changes in building 

performance after retrocommissioning were identified. And fifth, calibrated simulations 

of some of the buildings were performed to verify the effects of the identified changes on 

energy consumption. 

A preliminary group of 20 buildings which had been commissioned in 1996 or 1997 was 

initially selected. An office review of information on the retrocommissioning measures 

implemented and available information on operating parameters before and after 

retrocommissioning was then conducted. Based on this review, the 10 buildings with the 

most complete information concerning the retrocommissioning process and energy 

consumption data were selected. None of the buildings in this group received capital 

retrofits during the period 1996-2000. Five buildings were commissioned in 1996 and 

the other five were finished in 1997. In each of these buildings, commissioning measures 

were identified by the retro commissioning provider and then implemented by the 

provider, after receiving the concurrence of the building owner’s representative. Since all 

10 buildings were located on a university campus, they primarily consisted of classrooms, 

laboratories, and offices, with one volleyball arena. 

The energy usage data for these buildings had been monitored and was obtained 

beginning with the period shortly before retrocommissioning and ending in 2000 when 

the study was performed. For comparison purposes, all of the energy data was 

normalized to a single year of weather data. Because the weather data for the year 1995 

most closely approximated average weather conditions for the years studied, it was 

chosen as the baseline year. Energy use before and after the retrocommissioning process 

were compared. In this study savings from the retrocommissioning process were 

determined by using Option C of the International Performance Measurement and 

Verification Protocol, which determines savings using measured energy use at the whole 

facility level. This required that baseline models of the consumption be formulated for 

each major source of energy use in each building. Chilled water and hot water energy 

consumption were measured for each year, and three-parameter or four-parameter 

change-point models of cooling and heating consumption were determined as functions 

of ambient temperature using a modeling program. 

The process of calculating the yearly savings required the development of five separate 

chilled water models and five hot water models for each building, one for each year, 

including the baseline model. The consumption and savings for each year were then 

normalized to 1995 weather by using the models for each year's data with the 1995 

temperature data to determine the savings for each year. Electricity savings were 

determined without normalization since the buildings did not have chillers, and electricity 

consumption is not appreciably affected by ambient temperature. 

Follow-up was performed on two buildings with significant savings degradation. This 

was done primarily through a field investigation of the buildings to determine what 

changes had occurred that would produce the changes. Equipment performance and 

EMCS control settings were examined to evaluate possible causes for degradation. 

 36 
Information was then gathered on controls and operational changes that had occurred in 

the buildings during the period studied. This was done by examining the 

retrocommissioning reports and interviewing the engineers and maintenance personnel 

who had responsibility for each building. These interviews provided identifiable reasons 

for many of the changes in savings seen in the buildings. 

In order to quantify the effect of each operational or control change identified, it was 

decided that the energy usage of the buildings would be modeled using a computer 

simulation program. The rough simulations would then be calibrated until they provided 

accurate representations of the actual energy use. These simulations would then 

demonstrate how much of an effect each control or operational change had on the 

building energy use. 

Results 

All ten buildings showed significantly reduced chilled water and hot water energy 

consumption since retrocommissioning, although the savings generally decreased 

somewhat with time. Eight buildings had larger HW savings in 1998 than in 1997 as a 

consequence of hot water loop optimization conducted in 1997 and final 

retrocommissioning actions. Overall the electricity consumption remained fairly constant, 

with three buildings showing small increases in consumption (negative savings). The 

average electricity savings for the 10 buildings from 1997 to 2000 were 10.8 %. Figure 

2.1 and Figure 2.2 show the chilled water and hot water savings trends for the years 

following the building retrocommissioning. 

CHW Savings after Retrocommissioning

0

20

40

60

1997 1998 1999 2000

( % )

Eller

Harrington

Kleberg

Koldus

Richardson

VMC

Wehner

Zachry

Blocker

G. R. White

Figure 2.1. Chilled water savings persistence after retrocommissioning. 

 37 
HW Savings after Retrocommissioning

0

20

40

60

80

100

1997 1998 1999 2000

( % )

Blocker

Eller

Harrington

Kleberg

Koldus

VMC

Wehner

Richardson

G.R. White

Zachry

Figure 2.2. Hot water savings persistence after retrocommissioning. 

Overall, chilled water savings for the three years following retrocommissioning averaged 

39.3 % of the pre-commissioning baseline. Eight of the buildings showed good 

persistence of savings for chilled water (less than 15 % change during the 3 to 4 years 

after retrocommissioning), while the other two displayed significant degradation. The 

Blocker building had 19 % degradation, and the G. R. White Coliseum had a dramatic 

savings degradation of 38 %. 

Hot water consumption was reduced significantly in the years following 

retrocommissioning, but the savings fluctuated widely from year to year. Savings 

increased from 1997 to 1998 in most buildings due to optimization in the hot water loop 

in 1997 and some ongoing retrocommissioning work. The 10 buildings averaged hot 

water savings of 65.0 % after retrocommissioning. 

Based on the historic campus energy costs of $4,42/GJ ($4,67 /1 x 106

 Btu) for chilled 

water, $4,50/GJ ($4,75/1 x 106

 Btu) for hot water, and $0.02788/kW·h for electricity, the 

cumulative savings from retrocommissioning in these 10 buildings were $4 439 000 for 

the period 1997 - 2000. Only three buildings had year 2000 savings greater than 1998 

savings, and the increase in two of these was about 2 % of baseline consumption, which 

is well within the range of normal year-to-year variation. The savings of the other 

buildings decreased. 

Follow-up investigations of the two buildings with significant savings degradations 

revealed serious equipment malfunction and controls failure. In the Kleberg building, 

two chilled water control valves were found to be leaking badly, and combined with a 

failed electronic to pneumatic switch and high water pressure, caused low discharge 

temperatures and continuous reheat operation. In addition, failed sensors caused the 

outside air dampers to remain fully open, and leaking damper actuators in a number of 

 38 
VAV boxes resulted in simultaneous heating and cooling. The G. R. White Coliseum

was found to have a controls malfunction resulting in simultaneous heating and cooling, 

with two of the thirteen air handling units operating in heating mode, while the rest 

operated in cooling mode. These equipment and controls problems in these two buildings 

were the primary causes of the savings degradation observed. Because these problems 

did not result in comfort problems in the buildings, they may have gone undetected had 

the energy consumption not been monitored and compared with previous data. 

The energy management control system settings were evaluated for the buildings to 

determine why the changes in savings occurred. Three major control settings were 

examined: cold deck or cooling discharge temperatures, hot deck temperatures, and static 

pressure settings. The cold deck or cooling coil discharge temperatures were reset during 

retrocommissioning to save chilled water consumption. It was found that for eight of the 

ten buildings in 2000, the temperatures had been lowered and were requiring more 

cooling. This led to chilled water savings degradation, particularly in the Blocker 

building. Five of the ten buildings had dual duct systems, and of these five, three of the 

hot deck temperature set points were at different values in 2000 than they had been upon 

completion of retrocommissioning. This resulted in more hot water consumption. Static 

pressure set points affected chilled water, hot water, and electricity consumption. Of the 

nine buildings with variable air volume systems, only one (Koldus) still had the same

static pressure set point in 2000 that it had been set to during retrocommissioning. The 

other buildings were requiring more static pressure, and therefore using more energy. It 

is worth noting that the Koldus building showed no serious savings degradation of any 

kind in this study. 

Data were gathered from engineers and maintenance personnel to attempt to verify the 

controls changes and explain them. It was found that the G. R. White Coliseum, which 

saw significant savings degradation in chilled water and hot water savings, had 

experienced malfunctions in air handling unit controls that caused simultaneous heating 

and cooling to occur throughout the year. Almost all of the savings degradation for this 

building could safely be attributed to these problems. It was also found that the Kleberg 

building had experienced some significant equipment problems that could explain some

of the degradation in savings that occurred. No other building was reported to have 

experienced equipment problems of the same caliber as these two cases, but controls 

changes in the other buildings were verified through investigation. With the assembly of 

this type of information, simulated calibrations could be made for the buildings. Lack of 

data and other problems such as the one mentioned for the G. R. White Coliseum. White 

allowed only five of the ten buildings to be simulated. Three simulations were performed 

for each of these buildings, one for the pre-commissioning period, one for the year after 

retrocommissioning, and one for the year 2000. Factors considered in the simulations 

included control settings changes, operator overrides on the controls, and physical 

changes in the system such as broken or repaired valves, sensors, etc. 

Detailed simulations of the control changes in Eller O&M, Harrington Tower, VMC 

Addition and Wehner showed that the RMS difference between the changes observed 

between the post-commissioning periods and year 2000 was only 1.1 %, suggesting that 

 39 
the changes in savings for these buildings were almost entirely due to the control changes 

identified. 

Overall, equipment malfunction and changes made in cold deck and hot deck temperature 

settings following retrocommissioning were the major reasons for changes in chilled 

water and hot water energy consumption and savings after retrocommissioning. 

Table 2.1 is a summary of the money saved in the year 1998 as compared with the money 

saved in the year 2000 for each of the ten buildings examined. 

Table 2.1. Cost Savings Calculations for the Year 1998 and the Year 2000. 

Year 1998 Savings 

Savings Cost Savings

Year 2000 

No. Buildings Type Baseline Savings 

Energy Use 

(GJ/y)

Energy Use 

(GJ/y)

GJ/y Each $/y Total $/y Total $/y 

CHW 24 218 20 605 3 613 $ 15 993 

HW 9 216 1 768 7 448 $ 33 533 

1 Blocker

Elec. 4 832 3 883 950 $ 26 477 

$ 76 003 $ 56 738 

CHW 32 311 19 687 12 623 $ 55 875 

HW 8 001 1 218 6 783 $ 30 539 

2 Eller O&M

Elec. 4 891 3 675 1 217 $ 33 925 

$ 120 339 $ 89 934 

CHW 19 911 8 979 10 932 $ 48 386 

HW 22 319 580 21 740 $ 97 875 

3 G.R. White

Coliseum

Elec. 1 480 1 168 312 $ 8 712 

$ 154 973 $ 71 809 

CHW 14 959 8 883 6 076 $ 26 895 

HW 7 276 964 6 311 $ 28 413 

4 Harrington Tower

Elec. 1 666 1 336 330 $ 9 189 

$ 64 498 $ 48 816 

CHW 62 534 36 894 25 640 $ 113 491 

HW 43 059 1 281 41 777 $ 188 086 

5 Kleberg Building

Elec. 5 511 5 067 444 $ 12 380 

$ 313 958 $ 247 415 

CHW 23 173 13 703 9 470 $ 41 916 

HW 2 218 421 1 798 $ 8 093 

6 Koldus Building

Elec. 2 850 2 597 253 $ 7 067 

$ 57 076 $ 61 540 

CHW 30 096 16 497 13 599 $ 60 191 

HW 19 230 5 895 13 335 $ 60 035 

7 Richardson

Petroleum

Elec. 1 933 1 914 19 $ 519 

$ 120 745 $ 120 666 

CHW 43 143 25 406 17 738 $ 78 513 

HW 3 766 2 153 1 613 $ 7 260 

8 VMC Addition

Elec. 4 186 4 140 46 $ 1 286 

$ 87 059 $ 92 942 

CHW 20 249 14 073 6 177 $ 27 339 

HW 14 130 10 250 3 880 $ 17 469 

9 Wehner CBA

Elec. 2 555 2 446 109 $ 3 026 

$ 47 834 $ 68 145 

CHW 43 071 18 334 24 738 $ 109 496 

HW 8 098 3 408 4 690 $ 21 114 

10 Zachry Engr.

Center 

Elec. 7 502 6 793 710 $ 19 789 

$ 150 400 $ 127 620 

Type Totals Year 1998 Year 2000 

Chilled Water 313 666 183 062 130 605 $ 578 096 

Hot Water 137 314 27 940 109 374 $ 492 417 

Electricity 37 407 33 018 4 389 $ 122 371 

$1 192 884 $ 985 626 

* The baseline energy use data for two buildings were created based on the average savings of the other 

buildings because they did not have enough data. 

**To obtain MMBtu/yr, multiply the number of GJ/yr by 0.9478. 

 40 
Conclusions

Table 2.2 summarizes the savings history of this group of 10 buildings. The savings in 

1998 following initial retro commissioning corresponded to average energy cost savings 

of 39 % for the 10 buildings. Savings decreased to 32.3 % over the next two years – still 

a highly significant level of savings. 

Table 2.2. Summary of Savings History in 10 Retrocommissioned Buildings at Texas A&M 

Baseline Use

($/y) 

1998 Cx Savings 

($/y) 

Persistence of

Savings in 2000 

($/y) 

10 Buildings $3 049 487 $1 192 000 

(39.1 %) 

$984 516 

(32.3 %) 

8 Buildings $2 195 307 $723 376 

(32.9 %) 

$666 108 

(30.3 %) 

2 Buildings $854 180 $468 624 

(55 %) 

$314 408 

(37 %) 

Investigation showed that two of the buildings, G. R. White Coliseum and Kleberg, 

accounted for 3/4 of the total savings degradation, and both had experienced major 

equipment and controls malfunctions which were the primary causes of their degradation. 

Following correction of these problems, savings were restored to earlier levels. In the 

remaining eight buildings, savings changes were rather small, declining from 32.9 % to 

30.3 % in aggregate. 

All but one of the group of eight buildings had experienced at least some changes in 

EMCS control settings. To verify the impact of the EMCS changes on energy 

consumption, the calibrated simulation process was performed on the four buildings with 

the most complete data sets. Simulation was conducted for a pre-commissioning period, a 

post-commissioning period soon after retrocommissioning and for the year 2000 for each 

building. It was found that the changes in consumption observed following 

retrocommissioning in these buildings were consistent with those due to the identified 

controls changes, with an RMS difference of only 1.1 %. Control changes accounted for 

the savings increase observed in the Wehner Building as well as the decreases observed 

in the other three buildings. This suggests that the changes in savings these four were 

almost entirely due to the control changes. 

Based on the results of this study of 10 buildings, it was concluded that: 

• Basic retrocommissioning measures are quite stable, 

• Savings should be monitored to determine the need for follow-up, and 

• Steps should be taken to inform operators of the impact of planned/implemented 

control changes. 

 41 
8 Buildings in SMUD Program in Sacramento 

In 2003, a study was performed by the Lawrence Berkley National Laboratory (LBNL) 

on eight buildings that had undergone retrocommissioning through the Sacramento 

Municipal Utility District (SMUD) retrocommissioning program (Bourassa et al. 2004). 

The objective of the study was to determine the extent to which retrocommissioning 

measures were implemented, and the magnitude and persistence of energy savings 

achieved. Another objective was to see if the two primary goals of the SMUD 

retrocommissioning program had been met: reduced overall annual building energy 

consumption, and improved energy efficiency awareness and focus in the customer. The 

eight buildings selected for the study consisted of six office buildings, one laboratory, 

and one hospital. Four of the buildings participated in retrocommissioning in 1999, and 

the other four in 2000. In this program, the retrocommissioning provider worked with the 

building operators to develop the recommended measures. The measures selected for 

adoption were subsequently implemented by the building staff and/or contractors over a 

period of up to two years. 

Energy Analysis 

The energy savings obtained in the years following retrocommissioning were determined 

and compared. In order to be able to compare energy savings in the different buildings 

over the years examined, baseline energy consumption was established for each building 

based on pre-retrocommissioning energy use. Electricity use data were collected from

monthly utility bills for each building. Four buildings also had metered data recorded at 

15 minute intervals. Gaps in utility bills were filled from site records or regression 

analysis. 

The energy consumption data were normalized to a common weather year and to a 

common billing cycle of 30.5 days. The savings were calculated using spreadsheets, 

based on the normalized data, which allowed for a simpler and more robust statistical 

comparison. Another set of savings was also calculated, based on the 

retrocommissioning report predictions. Adjustments were made for a capital retrofit in 

one of the buildings. The cost of retrocommissioning was also estimated for each of the 

buildings, based on three categories: SMUD’s retrocommissioning costs, the site’s 

retrocommissioning costs, and the retrocommissioning measure implementation costs. 

Based on the estimated costs and savings, simple payback periods for 

retrocommissioning at each of the sites were calculated and compared. 

The electrical savings observed for each building over the years following 

retrocommissioning are shown in Figure 2.3. 

 42 
Source: Bourassa et al. 2004

Figure 2.3. Electrical savings following retrocommissioning for each of the buildings. 

The aggregate savings for the sites are shown in Figure 2.4. The buildings are grouped 

together according to the number of years of data available after retrocommissioning. 

Note that the “three year” line in the figure includes the data from the “four year” line 

plus data from three additional buildings, while the “two year” line simply adds data from

one more building. Comparison with the data in Figure 2.3 suggests that the peak in year 

3 may be largely due to the one building whose savings peaked in year 3. 

Source: Bourassa et al. 2004

Figure 2.4. Plot of aggregate post-retrocommissioning electricity savings. 

 43 
These plots demonstrate the observed trend in energy savings for the commissioned 

buildings. During the first two years the savings generally increased. This was expected 

because of the length of time needed for the retrocommissioning measures to be 

implemented. In the third year the savings began to level off, and the fourth year 

generally showed a declination in the electricity savings. A comparison with the 

predicted savings estimated in the retrocommissioning reports revealed that on average 

these reports underestimated the savings by 27.5 %. 

The average electricity savings for all the sites over all the years was 7.3 % per year. 

Natural gas usage could only be obtained for four of the buildings. The savings for 

natural gas were considerably lower, but since Sacramento is dominated by cooling 

needs, the lower natural gas savings only reduced the average total energy savings in 

these four buildings to 6.1 % per year. 

The payback periods for the retrocommissioning projects all proved to be attractive, with 

the longest period being 2.3 years. Table 2.3 lists the estimated costs, annual savings, 

and payback period for each site, as well as a price per square foot of the building. 

Table 2.3. Costs, energy savings, and payback periods for the eight sites studied. 

(Adapted from Bourassa et al. 2004)

Building RCx Study

Costs 

(Agent cost 

$25k, 

balance 

incurred by

site) 

Estimated

Measure 

Implmnt. 

Costs 

Predicted 

Avg 

Annual 

Savings 

($) 

Post-RCx 

Avg 

Annual 

Savings 

($) 

Predicted 

Simple

Payback 

Post-RCx 

Simple

Payback 

RCx 

Study

Costs 

($/m2

) 

RCx Study

& 

Implement. 

Costs 

($/m2

) 

Office 1 $28 000 $1 710 $24 500 $13 000 1.2 2.3 2.05 2.15 

Office 2 $26 500 $20 500 $21 900 $27 900 2.1 1.7 0.75 1.29 

Lab 1 $26 000 $12 370 $64 800 $40 100 0.6 1.0 3.01 4.41 

Hospital 1 $28 300 $11 180 $35 200 $30 900 1.1 1.3 1.18 1.61 

Office 3 $25 400 $150 $6 400 $22 400 4.0 1.1 0.65 0.65 

Office 4 $26 817 $8 380 $8 400 $22 600 4.3 1.6 0.86 1.18 

Office 5 $26 817 $4 350 $9 100 $15 800 3.4 2.0 0.86 0.97 

Office 6 $26 700 $3 000 $11 200 $48 600 2.7 0.6 0.97 1.08 

All Sites

Total 

$214 533 $61 650 $181 600 $221 200 1.5 1.2 0.97 1.29 

*To obtain costs in $/ft2, multiply the number of $/m2 by 0.0929. 

Measure Persistence Analysis 

A series of interviews and site visits were used to determine the persistence in the 

retrocommissioning measures recommended. The eight retrocommissioning reports 

recommended a total of 81 corrective measures, of which 48 were implemented. Of these 

48, it was found that 81 % had persisted, in that they were still in effect at the time of the 

study. It was discovered that four of the measures had been abandoned completely, all of 

which were air distribution component recommendations. Five of the measures had 

undergone evolution by the building engineers because the original measures had not 

resolved the problems. 

 44 
Surveys were given at the sites to determine attitudes regarding the retrocommissioning 

process, as well as its benefits. All of the sites reported that retrocommissioning was a 

worthy process. Four of the sites listed training as the primary non-energy benefit from 

retrocommissioning. The most cited downside to retrocommissioning was the time

intensive nature of the process. All of the sites came out of the retrocommissioning 

process with ideas on how to retain the commissioning benefits over time, the most 

common solutions being preventative maintenance plans. All of the sites would undertake 

retrocommissioning again, but only two had potential internal funding.

Conclusions

Some important retrocommissioning process factors that this study identified were: 

• The commissioning authority is most effective when he is both an expert and a 

teacher. 

• Building engineers prefer to evolve the settings on a recommendation that doesn’t

work, rather than revert to the previous condition. 

• Retrocommissioning appears to raise energy efficiency awareness. 

• Retrocommissioning funds are constrained within building management budgets. 

The energy analysis results showed: 

• Analyses should not emphasize first-year savings because savings typically take two 

to three years to fully manifest. 

• Energy savings persist to four years or more, although some degradation begins in the 

third year. 

• The retrocommissioning energy savings predictions were reasonably accurate. 

• Building managers lack tools for tracking energy performance. 

• Retrocommissioning cost pay back was shorter than the apparent savings persistence. 

• Retrocommissioning focused mostly on electricity savings and some natural gas trade 

offs in the savings occurred. 

The study suggested several recommendations for the SMUD Retrocommissioning 

program: 

• Develop measure implementation tracking agreements, possibly with inspections. 

• Explore methods to conduct a three year post-retrocommissioning energy 

consumption analysis using the billing history. 

• Develop simple Performance Tracking Tools for the building operators. 

• Develop an extension to the program whereby participants are eligible for new 

incentives in year 4 to evaluate and update the retrocommissioning as necessary. 

On the whole, the SMUD retrocommissioning program’s two broad goals were met at the 

eight sites. Aggregate post-retrocommissioning savings were strong, peaking at 

approximately 4 420 MW·h and the program helped educate site staff about energy 

efficiency and the role operations and maintenance plays. 

 45 
Oregon Case Study

A study performed in Oregon in 2004 examined eight Intel buildings that had been 

retrocommissioned in 1999 and 2000 (Peterson 2005). The buildings were located on the 

Intel Jones Farm and Hawthorn Farms campuses. Kaplan Engineering and PECI 

performed the retrocommissioning for these buildings through funding from Portland 

General Electric (PGE). At the time retrocommissioning occurred, it was estimated that 

electricity savings of nearly 3.5 million kW·h annually would result from the low cost 

energy efficiency measures (EEMs) proposed. The purpose of this study was to examine 

the energy usage of the buildings to determine what percentage of the original savings 

was still being achieved four years later. At the same time, it was desired to determine 

how many of the EEMs proposed were still being utilized. 

Three of the buildings studied were located on the Hawthorn Farms Campus, and were 

designated HF1, 2, and 3. The buildings combined for a total of 59 457 m2

 (640 000 ft2), 

and were served by a central chiller and boiler plant. HF1 had DDC control interfaced 

with pneumatic actuators, and the other two buildings were upgraded to DDC control in 

2000. The remaining five buildings studied were located on the Jones Farm Campus, and 

were designated by JF. They combined for a total of 130 063 m2

 (1 400 000 ft2), with 

over 40 major air handling systems served by two central chiller plants and two hot water 

boiler plants. Most of the spaces on both campuses were served by variable air volume 

(VAV) systems. 

Three reports generated at the time of retrocommissioning were examined to determine 

what measures had been implemented. The current status of these measures was 

determined through random sampling, with functional testing or trending being used as 

appropriate. For HF1, the terminal reheat units were serviced at the time of 

retrocommissioning to ensure proper damper motion. At the time of this study, random 

sampling discovered no noticeable damper movement from full cooling to full heating in 

60 % of the units. The savings for this measure did not persist, probably due to the aging 

pneumatic system. For HF 1, 2, and 3, retrocommissioning had modified outside air 

intake controls to allow for the economizing cycle to function. At the time of the study, 

random sampling revealed this measure to still be functioning. For the HF chillers, 

retrocommissioning had lowered the condenser water set point from 23.9 o

C (75 oF) to 

21.1 oC (70 o

F), while raising the chilled water set point from 5.6 oC (42 oF) to 7.2 oC (45 o

F). This measure was also found to be in operation at the time of this study. 

For the Jones Farm buildings, air handling units and terminal boxes were scheduled at the 

time of retrocommissioning to reflect occupancy patterns, scheduling unoccupied hours 

as 6 PM to 6 AM on weekdays and all day on weekends. At the time of this study, JF3 

was evaluated, and the control was found to be working fairly well, with only a couple of 

override issues. Additional savings opportunities for the JF buildings were also identified 

in this study, including air flow and scheduling opportunities and control overrides that 

needed adjustment. For the HF chillers, the leaving condenser water set point was 

lowered from 23.9 o

 C (80 oF) to 23.9 o C (75 oF) 67 oF at the time of retrocommissioning. 

The current study found the set point to be at 71o

F, still significantly lower than the 

original. 

 46 
Overall at the Hawthorn Farms campus the ECMs were found to have been maintained, 

with the exception of the terminal unit reheat optimization in HF1. Of the original 

projected savings in the three buildings at Hawthorn Farms, 89 % of the electric savings 

and 0 % of the natural gas savings were still being achieved at the time of this study. In 

the five buildings at Jones Farm, the results were more mixed and less quantifiable. The 

recommended scheduling changes were still programmed at a high level, but it appeared 

that numerous control overrides at a zone or box level had been made. Some overrides 

may have been due to changes in space use (such as conversion to a lab), but in many 

instances conference and training rooms were maintaining occupied modes around the 

clock. The trending done on some of the variable speed air handlers showed little 

difference between day and nighttime airflow suggesting that terminal box scheduling 

was not having an impact on overall airflow. 

Summary

Of the eight buildings retrocomissioned in Oregon in 1999 and 2000 quantitative findings 

were reported for three and qualitative findings for the other group of five buildings. For 

the three buildings on the Hawthorn Farms campus, totaling 60 000 m2

 in floor area, 89 

% of the original electric savings were achieved in 2004 and 0 % of the natural gas 

savings were achieved in 2004. For the five buildings on the Jones Farm campus with 

130 000 m2

 of floor area, the results were mixed and less quantifiable. It was found that 

scheduling changes were still programmed at a high level, but numerous control 

overrides at a zone or box level had been made. 

Office Building in Colorado 

A study completed in 2005 evaluated the persistence of recommissioning savings in a 

large office building in Colorado (Selch and Bradford 2005). Of the studies of this kind 

done to date, this study appears to have chosen the largest window of time over which to 

look at persistence. The office building was recommissioned in 1995, which resulted in 

verified savings of 14 % in electrical demand, 25 % in electrical use, and 74 % in gas use. 

In 2003, the building was again recommissioned, at which time the status of the energy 

conservation measures implemented in the initial recommissioning effort was evaluated. 

The computation of savings was done in two ways. The overall energy use of the 

building for each year was obtained from utility bills. These data were then normalized 

to account for factors such as weather differences, changing occupancy patterns in the 

building, and added construction in the building. In this way the yearly energy use could 

be accurately compared to the baseline, pre-commissioned energy use. The other savings 

calculation method was an individual measure evaluation. Specific measures that 

impacted individual HVAC system components were examined. To perform the 

calculations, Options B & C of the International Performance Measurement and 

Verification Protocol (IPMVP 2001) were employed, Option B being used for individual 

measure evaluation, and Option C for whole building usage comparison. 

Table 2.4 summarizes the results of the individual measures evaluation. The savings 

from the 2003 recommissioning effort are compared with the 1996 savings. To 

determine the persistence of savings, the percentage of 1996 savings achieved after 

 47 
recommissioning in 2003 was subtracted from 100 %. This is because it was supposed 

that the difference in achieved savings between the two recommissioning efforts 

represented those savings that had persisted. 

Table 2.4. Electric savings persistence summary. 

1996 Savings 2003

Electricity 20 %

(1 600 000 kWh) 

83 % Persistence 

(17 % Savings) 

(1 330 000 kWh) 

Demand 14 %

(219 kW) 

86 % Persistence 

12 % Savings 

(188 kW) 

Gas 74 % Complete persistence 

As noted in Table 2.4, it was calculated that 86 % of the electrical demand savings had 

persisted, while 83 % of the electrical use savings had persisted. The results of the whole 

building energy use comparison appear in Source: Selch and Bradford 2005 

Figure 2.5 and Source: Selch and Bradford 2005 

Figure 2.6. The left chart in each figure represents the raw values, while the right chart 

displays adjusted, normalized values. 

Source: Selch and Bradford 2005

Figure 2.5. Annual electrical demand, raw and adjusted. 

 48 
Source: Selch and Bradford 2005

Figure 2.6. Annual electrical use, raw and adjusted. 

The annual demand and consumption values that were adjusted to account for changing

conditions indicated that the savings achieved from recommissioning had largely 

persisted. This was concluded with greater confidence due to the corroboration of the 

independent measure analysis. 

The study reported that a large majority of the energy savings measures implemented in 

the original recommissioning effort had persisted, as had their resultant energy savings. 

This was in spite of changing conditions in the building, including a complete change in 

operation staff. It was concluded that ECMs of this nature can persist for at least eight 

years even with limited support from operators and staff. However, it was noted that 

continued, on-going support to the building staff as part of the original recommissioning 

effort probably would have resulted in complete persistence of the savings achieved.

III. Persistence of Commissioning Measures in New Buildings 

PECI PIER Study 

In the summer of 2002, a study was completed that had begun in the fall of 2001 under a 

California Energy Commission Public Interest Energy Research (PIER) project 

(Friedman et al. 2003b). The purpose of the study was to examine ten buildings that were 

commissioned at building start-up in order to address the persistence of benefits from the 

commissioning process. This study drew qualitative conclusions about the persistence of 

new building commissioning, focusing on three issues: how well the benefits of 

commissioning persisted, the reasons for declining performance, and the methods that 

can be used to improve the persistence of benefits achieved through commissioning. A 

quantitative assessment of persistence by measure (“this measure has an expected 

persistence of X years”) was outside the scope of this project, since a large number of 

buildings would have been required to determine the figures for each measure. 

To evaluate the persistence of commissioning benefits on new buildings, the buildings 

first had to be selected. To qualify for the study, the facility needed to have been 

 49 
commissioned as a new building or major retrofit between two and eight years prior to 

the study. Due to the difficulty in finding such buildings with adequate commissioning 

documentation in California, five buildings were selected in the Pacific Northwest, and

five more in California. It was not feasible to limit the study to buildings that followed 

the full commissioning process, from pre-design through final acceptance and postoccupancy, as described in ASHRAE Guideline 1 (ASHRAE 1996). The most 

completely commissioned and documented buildings were sought, but these typically did 

not include design-phase commissioning. 

For each building, three to eight items were identified that were documented to have been 

fixed during commissioning. The changes and repairs made during commissioning 

generally fell into three categories: hardware, control system, and documentation 

improvements. Due to the focus on energy savings measures in the study, the hardware 

and control system changes with the greatest energy implications were of highest interest, 

as well as measures dealing with comfort and reliability. The amount of documentation 

available for each measure was also a driving force in measure selection. It was 

necessary to only evaluate those measures that had actually been implemented and 

documented. Routine maintenance issues or measures deemed static once corrected 

(such as equipment disconnected from the power supply) were not looked at. With the 

limited amount of time and funding for the study, it was necessary to focus on measures 

whose current status could easily be compared to the as-commissioned status and which 

would affect energy consumption. Because of the bias in selecting these measures, and 

the underestimation of savings persistence due to the limited number of measures 

considered, the results of the study were presented qualitatively. 

For purposes of the study, it was decided that if the measure resulted in better 

performance than the pre-commissioning condition, then the measure was said to have 

persisted, even if it had been adapted to meet real operating conditions of the building. In 

some cases the persistence of a measure was somewhat subjective. 

The people with the most knowledge about the control system at each site were 

interviewed. Some sites were identified for site visits, and for the others a second 

interview was conducted to discuss the current status of the commissioning measures. 

Six of the buildings were visited, during which the persistence of the selected 

commissioning measures was investigated, and the work environment and resources 

available to the operations staff were evaluated. 

Results 

It was found that the process of finding qualified buildings for the study in California was 

difficult. As mentioned above, qualified buildings were located more easily in Oregon, 

most likely because of the longer history of new building commissioning in the Pacific 

Northwest. California had numerous existing buildings involved in retrocommissioning 

projects, but new buildings having undergone commissioning at least two years earlier 

were sparse. For many of the commissioned buildings considered for the study, 

commissioning reports had not been written, so the information that could have been used 

by operations personnel to more efficiently operate the building essentially was lost. 

Often times in lieu of a report, the commissioning activities would simply be placed on a 

 50 
“punch list” for maintenance personnel to work on, who, when they had completed them

usually did not document the changes. In other buildings the reports had been written, 

but were not readily available to the operations staff, having been filed away in storage 

and not easily accessible. In many cases where documentation did exist, it was not clear 

when or if the commissioning measures had been implemented, as they were noted as 

“recommendations” or “pending.” These issues led to the conclusion that the term

“commissioning” had been applied to a variety of different activities, including 

troubleshooting items and checklists, indicating a lack of consistency in the way the term

was being applied. 

Table 2.5 summarizes the commissioning measures studied and their level of persistence. 

A light gray square indicates that the measure persisted, while a black square indicates

that the measure did not persist. A square split in half horizontally indicates that more 

than one measure was investigated in the category. 

Table 2.5. Persistence of equipment and controls fixed during commissioning. 

Source: Friedman et al. 2003b

Com

missioning report on site

Com

missioning report used

Control sequences availableChiller control Cooling tower controlBoiler control

 Hydronic control

 Economizer control algorithm

Discharge air te

mperature reset 

 Simultaneous heating and cooling

VFD modulation 

 Dessicant cooling

Duct static pressureSpace te

mperature control

Terminal units Piping and fitting proble

ms

Valve modification

Wiring and instrumentation

Sensor placement or addition

Sensor error or failureScheduling

 Skylight louver operation

Occupancy sensor

 - yes

Office Building 1 (1996) no - yes

Office Building 2 (1996) no - no

yes yes no

Office Buidling 4 (1994) no - 

no - yes

Medical Facility 1 (1998) yes yes yes

yes yes yes

 - yes

Lab and Office 3 (2000) no - no

California Pacific Northwest

BUILDING 

(year commissioned)

CENTRAL 

PLANT

AIR HANDLING AND 

DOCUMENTS DISTRIBUTION

no

Lab and Office 2 (1997)

Medical Facility 2 (1998)

Office Building 5 (1997)

no

OTHER

PREFUNCTIONAL

TEST

Lab and Office 1 (1995)

Office Building 3 (1994)

 

Across the ten buildings studied, patterns about the types of commissioning fixes that 

persisted emerged. For the fifty-six commissioning fixes selected, well over half of the 

measures persisted. It was not surprising that hardware fixes, such as moving a sensor or 

adding a valve, persisted. Furthermore, when control algorithm changes were 

reprogrammed, these fixes often persisted, especially when comfort was not 

compromised. Many design phase fixes may have persisted in a similar way, but these 

were not able to be studied since only one building was commissioned in the design 

phase. 

 51 
The types of measures that tended not to persist were the control strategies that could 

easily be changed, such as occupancy schedules, reset schedules, and chiller staging. 

Four out of six occupancy schedules did not persist. Chiller control strategies did not 

persist in three out of four cases, most likely due to the complex nature of control in 

chilled water systems. The study of sensor issues was limited to major sensor problems 

that were corrected during commissioning, such as sensor failure or excessively faulty 

readings. With this selection bias applied, two out of five sensor repairs did not persist. 

Among the commissioning measures implemented, a few cases involved technologies 

that were new or different from normal practice. Due to lack of documentation, these 

measures were not included in this study, but it was observed during the investigation 

that these measures generally did not persist. This was attributed to a lack of operator 

training for the technologies. 

Discussion 

The study suggested three possible reasons for lack of persistence among some measures. 

The first was limited operator support and high operator turnover rates. Operators often 

did not receive the training necessary or they did not have sufficient time or guidance for 

assessing energy use, and the training given new operators who came in after the 

commissioning was usually inadequate. The second reason involved poor information 

transfer from the commissioning process. For nearly every case studied, the 

commissioning report was either difficult to locate, or was not even located on site, which 

reduced the ability of building operators to review commissioning measures 

implemented. The third reason for lack of persistence was a lack of systems to help track 

performance. Operators spent most of their time responding to complaints and 

troubleshooting problems, leaving little time to focus on assessing system efficiency. 

Aside from this, lack of information and knowledge impeded the efficiency assessment 

by building operators. 

The persistence of commissioning benefits was found to be highly dependent on the 

working environment for building engineers and maintenance staff. A working 

environment that was supportive of persistence included adequate operator training, 

dedicated operations staff with the time to study and optimize building operation, and an 

administrative focus on building performance and energy costs. Trained operators were 

found to be knowledgeable about how the systems should run and, with adequate time

and motivation to study the system operation, these operators evaluated and improved 

building performance. In five buildings, operators participated in the commissioning

process and came away with a good understanding of their systems. In addition, good 

system documentation in the form of a system manual served as a troubleshooting 

resource for operators at two buildings. It was noted that administrative staff can help 

enable a supportive working environment by placing high priority on energy efficient 

systems and operator training. Only a few of the buildings studied seemed to operate in 

this environment, and the measures investigated at these facilities had the highest rate of 

persistence. 

 52 
Some of the measures simply persisted by default – no maintenance being required to 

keep them operational. If comfort issues were not a factor, or the measure involved 

programming buried deep within code, the measures tended to persist. 

The study recommended four methods for improving persistence. First, operators should 

be provided with training and support. Especially with high operator turnover, adequate 

training is needed for benefits to persist, and a working environment with energy 

efficiency as a high priority is also beneficial. Second, a complete systems manual 

should be provided at the end of the commissioning process. This will serve as a 

reference for building operators, and will allow the systems knowledge gained from the 

commissioning process to be available over the long term. Third, building performance 

should be tracked. New building commissioning efforts should help to implement 

mechanisms for performance tracking, including what information to track, how often to 

check it, and the magnitude of deviations to address. Fourth, commissioning should 

begin in the design phase to prevent nagging design problems. Changes made on paper 

before construction has begun tend to be more cost effective and have higher levels of 

persistence. 

The study concluded with a recommendation that more in-depth, quantitative studies be 

performed to investigate the life of commissioning measures and carry out cost-benefit 

analyses for new building commissioning. It was further recommended that a manual of 

guidelines for improving persistence be developed to give guidance and direction to 

building operators with regard to energy efficiency. 

 IV. Strategies for Improving Persistence in New and Existing 

Buildings 

As a follow-up to the study of persistence of commissioning benefits for new buildings 

performed in California and Oregon, and the study of persistence of retrocommissioning 

benefits done at Texas A&M (both described previously), a report was issued in July 

2003 addressed to building owners, managers, and operators suggesting methods for 

improving the persistence of commissioning benefits for both new and existing buildings 

(Mills et al. 2004). The report began by summarizing the key conclusions of both 

studies, namely that many commissioning benefits tend to persist fairly well, but that

significant opportunities still exist for improving overall savings persistence. The report 

then proposed that an emphasis on certain key elements of energy analysis and efficiency 

would pave the way for long-term success in building operation and energy use. In 

particular, seven recommendations were discussed at length: design review, building 

documentation, operator training, building benchmarking, energy use tracking, trend data 

analysis, and recommissioning. A summary of the discussion of each of these topics is 

presented below. 

Design Review

As many as one-third of major commissioning problems can be traced back to the design 

phase of the project and these problems often plague building operators throughout the 

life of the building. Allowing professional engineers to review the design while still in 

the design phase of the project is a cost-effective way to prevent future problems. 

 53 
Correcting design problems on paper is easier and less costly than attempting to correct 

them once the building is completed. Some of the issues to be considered in reviewing a 

design are test port location, equipment accessibility, load calculations and minimum 

flow settings, control system sequences and point lists, and standard design details. The 

process of design review should begin as soon as possible to allow opportunity for 

correction. 

Building Documentation 

Good system documentation is not a common practice currently in the construction 

environment. While it may seem like a costly and time-consuming effort, this 

documentation is the best way to ensure that the knowledge base obtained during design, 

construction, and commissioning of the building is preserved, and will aid in maintaining 

commissioning benefits. The three most vital items to document are the final design

intent, the sequences of operation, and the system diagrams. Other important documents 

include the operator’s log, commissioning summary report, general description of facility 

and systems, as-built documents, detailed description of each system, location of all 

control sensors and test ports, and capabilities and conventions of the DDC system. The 

best time for this documentation to occur is during the construction phase of the building. 

For existing buildings, a good time is during a retrofit or recommissioning. The 

documentation should be compiled into a systems manual that is readily accessible. 

Operator Training 

Effective operator training will allow the benefits of building commissioning to persist, 

and will aid in preventing problems. Training opportunities exist for building operators 

during the commissioning process, through manufacturers and vendors, in operator 

certification programs, and using building documentation. It is also essential that new 

operators be trained sufficiently so that the knowledge gained by the previous operator is 

not lost. Some suggested training topics include: descriptions of equipment, equipment 

start-up and shut-down procedures, operation and adjustment of controls, review of 

system documentation, common troubleshooting problems, maintenance requirements 

and schedules, health and safety issues, special tools and spare parts inventory, and 

emergency procedures. 

Building Benchmarking 

Benchmarking refers to measuring the energy use of a building relative to other 

buildings, and provides a way to track energy use over time and compare it with the 

competition. This will allow building owners and operators to prioritize initiatives and 

improve energy efficiency. Several tools exist to aid in the benchmarking process. Two 

of these are the ENERGY STAR Portfolio Manager, which uses a number of factors to 

make meaningful comparisons with other buildings under different conditions, and the 

Cal-Arch Building Energy Reference Tool, which is a quick and simple tool for 

comparing energy use per square foot. 

 54 
Energy Use Tracking 

Tracking utility bills or metered data is an effective method for recognizing energy use 

problems that may not result in comfort problems, and therefore might not be noticed any 

other way. It is essential for continued energy efficiency and persistence in

commissioning benefits. The energy use curves should be compared for different years 

to look for patterns, anomalies, and peaks and valleys. An Energy Information System is 

a useful tool for automating utility tracking. It saves time, provides immediate feedback, 

can gather additional data, and can allow access over the Internet. 

Trend Data Analysis 

DDC systems allow points to be trended over time. Knowing how to interpret these 

trended data is essential for identifying and correcting problems in building energy 

consumption and performance. The data should be examined regularly to determine if

the system and its individual components are functioning as desired. Automated 

diagnostic tools exist to aid in this process, having automated capabilities in the following 

categories: data acquisition, archiving and pre-processing, detection, and diagnosis. Two 

tools available are ENFORMA and PACRAT. PACRAT can be used as an ongoing 

diagnostic tool. 

Recommissioning 

The process of recommissioning, especially when it draws on building documentation 

and previous commissioning activities, is very effective in maintaining commissioning 

benefits. The time to consider recommissioning largely depends on the effectiveness of 

operations and maintenance strategies and overall building performance. Commissioning 

can be performed by an outside commissioning provider when an outsider’s view is 

considered helpful, or it may be done in house. In-house commissioning increases the 

knowledge level of those participating with regards to building operation. Continuous 

Commissioning is an ongoing commissioning process developed by the Energy Systems 

Laboratory at Texas A&M University that has the same general goal as 

retrocommissioning, but focuses strongly on the persistence of commissioning benefits. 

The report concluded by reiterating the need to pursue the topics addressed during and 

after the commissioning process to maintain the benefits of commissioning over the longterm. 

V. Related Reports 

A report was compiled in 2004 that evaluated the cost effectiveness of commissioning in 

new and existing buildings (Mills et al. 2004). The largest study of its kind to date, it 

examined the results of commissioning for 224 buildings across 21 states. Among the 

existing buildings commissioned, a median payback period for commissioning was 

reported to be 0.7 years. For new buildings, this value was found to be 4.8 years. Both 

of these figures excluded non-energy benefits, which would increase the savings 

experienced.

 55 
While persistence of savings was not the primary focus of the study, it was examined 

briefly since it plays a role in determining overall savings. Figure 2.7 shows the 

persistence of savings results for 20 of the buildings in the study, with a four year period 

following commissioning in each building. The savings are indexed by a comparison of 

the year’s consumption to the pre-commissioning baseline consumption. The savings are 

compared by category: electricity, fuel, chilled water, and steam/hot water. 

Source: Mills et al. 2004 

Figure 2.7. Emergence and persistence of energy savings (weather normalized). 

An important factor noted in the report was the fact that in many cases of commissioning, 

the recommended measures were implemented gradually; indicating that the first year 

after commissioning was not the best year for calculating savings. On the other hand, it 

was also observed that after time some of the savings began to degrade due to changing 

building conditions, operations, or aging. As seen in the figure, the maximum value for 

savings was reached and subsequently savings began to degrade. This effect was 

smallest for electricity, but much more noticeable for chilled and hot water and steam. 

With regard to persistence of commissioning benefits, the report concluded that tracking 

energy consumption for evidence of significant consumption increases is the most 

important means of determining the need for follow-up commissioning, and that while 

controls changes by building operators account for a portion of savings degradation, 

hidden component failures are perhaps the greatest culprit in persistence problems. 

VI. Methodologies for Determining Persistence of 

Commissioning Measures and Energy Benefits of 

Commissioning 

The retrocommissioning studies that provided a quantitative evaluation of the persistence 

of energy benefits of commissioning used variations on two different approaches to 

evaluate the persistence of energy benefits. 

 56 
The study of 10 Texas buildings (Turner et al. 2001) used a variation on Option C of the 

IPMVP that normalized for weather differences between years by selecting a “normal” 

year of weather data in the sequence available that most closely met long-term norms. A 

suitable three-parameter or four-parameter regression model of the baseline year was 

created along with models of the performance of the building in each year evaluated. 

Then the annual consumption for each year was determined by running the appropriate 

model with the appropriate year of weather data. The study of eight SMUD buildings 

(Bourassa 2004) used the same methodology, except that they used a long-term average 

weather year instead of selecting one of the available years of weather data. The 

Colorado study used a different approach, evaluating savings persistence with IPMVP 

Option C with baseline adjustments and IPMVP “Option B” was used to determine 

savings for specific measures in operation. The Oregon study did not specify how savings 

were evaluated. 

The study of eight buildings in Oregon (Peterson 2005) and the Colorado building (Selch 

and Bradford 2005) used a different approach. These studies examined each of the 

measures that had been implemented and determined whether the measures were still in 

place and functioning. Peterson found that in three of the buildings, she could quantify 

the savings associated with measures that had been disabled after four years. It was 

found that numerous measures implemented in the other five buildings were still in place, 

but there were also numerous overrides and changes that had occurred as well. It was not 

possible to quantify the degree of persistence in these buildings. Selch and Bradford 

(2005) found that they were able to quantify the savings associated with measures that 

had been disabled. 

The study of 10 new buildings that had been commissioned in Oregon and Washington 

(Friedman et al. 2003b) used a methodology that quantified the number of measures that 

were still in place, but it did not seem appropriate to try to quantify the energy savings 

associated with these measures. The four retrocommissioning studies all discussed the 

measures found to be still operating and those that had been changed. The Texas study 

used calibrated simulation to evaluate measures that had been changed. The other studies 

were not explicit in the methods used to evaluate the impact of measure changes. 

VII. Summary and Conclusions 

The results of studies from five projects related to commissioning, either in new or 

existing buildings, described above represent the extent of research that has been 

performed with regard to the persistence of commissioning benefits over time. 

The savings in the buildings that were retrocommissioned generally showed some

degradation with time. In retrocommissioned buildings, savings generally decreased with 

time, but there is wide variation from building to building. For the buildings where 

savings persistence was quantified:

• Savings persistence at the time of the study (3 to 8 years after commissioning) ranged 

from about 50 % to 100 % in all but one or two buildings. 

 57 
• Average savings at the time of the study were about 3⁄4 of the original savings. 

• The most dramatic savings take-backs were caused by undetected mechanical or 

control component failures. 

For the new buildings, well over half of the 56 commissioning fixes persisted. Hardware 

fixes, such as moving a sensor or adding a valve, and control algorithm changes that were 

reprogrammed generally persisted. Control strategies that could easily be changed, such 

as occupancy schedules, reset schedules, and chiller staging tended not to persist. It was 

also found that the extent to which persistence occurs is also related to operator training. 

As is evident, the number of buildings studied in all of the papers described here 

represents a very small portion of commercial buildings that have undergone 

commissioning or retrocommissioning. More research is needed to: 

• Develop a uniform methodology for determining commissioning persistence. 

• Determine the persistence of savings from a broader sample of buildings. 

• Develop simple tools for tracking performance of commissioning measures. 

• Develop practical methods for owners and operators to better maintain 

commissioning savings. 

References 

Bourassa, N.J., M.A. Piette, N. Motegi., “Evaluation of Persistence of Savings from

SMUD Retrocommissioning Program – Final Report,” Lawrence Berkeley National 

Laboratory, May 2004. LBNL-54984 

Chen, Hui, Song Deng, Homer Bruner, David Claridge and W.D. Turner, “Continuous 

CommissioningSM Results Verification And Follow-Up For An Institutional Building – A 

Case Study,” Proc. 13th Symposium on Improving Building Systems in Hot and Humid 

Climates, May 20-23, 2002, Houston, TX, pp. 87-95. 

Cho, Sool Yeon, “The Persistence of Savings Obtained from Commissioning of Existing 

Buildings,” M.S. Thesis, Mechanical Engineering Department, Texas A&M University, 

ESL-TR-02-05-01, May, 2002, 347 pp. 

Claridge, D.E., Turner, W.D., Liu, M., Deng, S., Wei, G., Culp, C., Chen, H. and Cho, 

S.Y., “Is Commissioning Once Enough?,” Solutions for Energy Security & Facility 

Management Challenges: Proc. Of the 25th WEEC, Atlanta, GA, pp. 29-36, Oct. 9-11, 

2002. 

Claridge, D.E., Turner, W.D., Liu, M., Deng, S., Wei, G., Culp, C., Chen, H., and Cho, 

S.Y., “Is Commissioning Once Enough?” Energy Engineering, Vol. 101, No. 4, 2004, pp. 

7-19. 

Friedman, H., Potter, A., Haasl, T., and Claridge, D., “Persistence of Benefits from New 

Building Commissioning,” Proceedings of the 2002 ACEEE Summer Study on Energy 

Efficiency in Buildings, Pacific Grove, CA, Aug. 19-23, 2002, pp. 3.129 – 3.140. 

 58 
Friedman, H., Potter, A., Haasl, T., Claridge, D and Cho, S., “Persistence of Benefits 

from New Building Commissioning,” Proc. Of 11th National Conference on Building 

Commissioning, Palm Springs, CA, May 20-22, 2003a, 15 pp., CD. 

Friedman, H., A. Potter, T. Haasl, and D. Claridge, “Report on Strategies for Improving 

Persistence of Commissioning Benefits – Final Report,” July 2003b, 47 pp. Lawrence 

Berkeley National Laboratory. http://buildings.lbl.gov/hpcbs/pubs/E5P22T5c-Final.pdf

IPMVP 2001. IPMVP Committee, International Performance Measurement & 

Verification Protocol: Concepts and Options for Determining Energy and Water Savings, 

Vol. 1, U.S. Dept. of Energy, DOE/GO-102001-1187, 86 pp., January. 

Liu, C., Turner, W.D., Claridge, D., Deng, S. and Bruner, H.L., “Results of CC FollowUp in the G. Rollie White Building,” Proc. 13th Symposium on Improving Building 

Systems in Hot and Humid Climates, May 20-23, 2002, Houston, TX, pp. 96-102. 

Mills, E., H. Friedman, T. Powell, N. Bourassa, D. Claridge, T. Haasl, and M. Piette, 

“The Cost-Effectiveness of Commercial-Buildings Commissioning: A Meta-Analysis of 

Energy and Non-Energy Impacts in Existing Buildings and New Construction in the 

United States,” December 2004. LBNL-56637. 

Mills, E., N. Bourassa, M.A. Piette, H. Friedman, T. Haasl, T. Powell, and D. Claridge. 

"The Cost-Effectiveness of Commissioning New and Existing Commercial Buildings: 

Lessons from 224 Buildings," Proceedings of the 2005 National Conference on Building 

Commissioning, Portland Energy Conservation, Inc., New York, New York, May, 2005. 

Peterson, Janice, “Evaluation of Retrocommissioning Results After Four Years: A Case 

Study,” Proceedings of the 2005 National Conference on Building Commissioning, 

Portland Energy Conservation, Inc., New York, New York. 

Selch, M. and J. Bradford, “Recommissioning Energy Savings Persistence,” Proceedings 

of the 2005 National Conference on Building Commissioning, Portland Energy 

Conservation, Inc., New York, New York, May, 2005.

Turner, W.D., Claridge, D.E., Deng, S., Cho, S., Liu, M., Hagge, T., Darnell, C., Jr., and 

Bruner, H., Jr., "Persistence of Savings Obtained from Continuous CommissioningSM,"

Proc. of 9th National Conference on Building Commissioning, Cherry Hill, NJ, p. 20-1.1 

- 20-1.13, May 9-11, 2001. 

 59 
Chapter 3: Review of Automated Commissioning Tools 

for Buildings 

I. Introduction 

Commissioning is a quality control process developed to ensure that building systems 

operate as intended and meet their design intent. Commissioning is also an opportunity 

to achieve greater occupant comfort and energy-efficiency. As the value of 

commissioning is increasingly recognized, so is the need to facilitate this process. 

Researchers have worked on various approaches to aspects of this problem over the past 

decade; however, the availability of fully automated commissioning tools is limited. At 

present, the state of the art includes a number of automated and semi-automated 

commissioning tools that are being developed and tested at research institutions and 

universities with funding from utilities, industry, and government agencies. It is 

important to note that this review is based on publicly available information and therefore 

does not include tools in development by individual controls companies. As these 

technologies are proven, it is anticipated that the building control industry and energy 

services industry will be a key source as well as user of automated tools. 

This chapter summarizes findings from a review of existing automated commissioning 

tools for buildings and related research. It builds on the Brambley et al. (2003) report on 

the use of automated tools for building commissioning. The second section gives an 

overview of the main tools to be discussed and includes tables that provide a more 

detailed description of the tools/prototypes and the key functions they are designed to 

perform. Because market availability of such tools is very limited, several prototype 

tools are also included. Section three takes a broader look at related research, including 

several enabling tools, and provides key references for work in the area of fault detection 

and diagnostics because of its relevance to continuous commissioning and the persistence 

of commissioning benefits. Finally, the chapter concludes with an overview of the 

barriers to automated commissioning and some recommendations for future work. 

Images of the tools discussed in this chapter can be found in the Appendix to this report. 

II. Automated Commissioning Tools 

The automated commissioning tools developed to date can broadly be categorized as: 

• Tools to evaluate the performance of systems, or 

• Tools that automate other aspects of the commissioning process. 

These tools, which have been developed for a variety of conventional HVAC systems,

address various aspects of initial commissioning and recommissioning/ongoing 

commissioning. Furthermore, they have been developed for different end-users in mind. 

The first type, tools for performance evaluation, can be classified into tools for passive 

testing and for active testing. Passive testing involves using the control system to 

monitor and record sensor readings and control signals. It is non-invasive, monitoring 

 60 
the system under normal operating conditions. Data analysis methods are then applied to 

the collected data. Active testing involves injecting test signals that artificially change 

the system operation so as to exercise the system over its operating range. It is a means 

to extract a greater amount of information over a shorter time than in the case of passive 

testing but is invasive and more expensive to implement (Visier et al. 2005). 

A common thread among many of the tools is the use of energy management and control 

systems (EMCS) or building automation system (BAS) for various functions, ranging 

from providing historical data to real-time control information. Building control systems 

have the capability to collect and store large amounts of time-series data; however, its use 

in that format has been quite limited because of the difficulty of extracting useful 

information. In response, open protocols such as BACnet greatly increased in the 1990’s 

and diagnostic tools emerged in the late 1990’s providing varying levels of monitoring 

capabilities. Friedman and Piette (2001) presented a review of emerging diagnostic tools 

that used EMCS data in either a manual or automated tool. Since that time, researchers 

have continued to pursue the automation of diagnostics, developing a number of tools 

that use data from the EMCS. And although the increasing availability of EMCS over the 

last ten years has resulted in greater access to operational data and a means to manipulate 

system operation, data-handling schemes are not standardized. In most cases, data 

configuration must be developed for each individual project, which is expensive. 

Tool Selection 

Although all of the commercially-available automated commissioning tools discussed in 

this review are US products, international research on automated commissioning has 

increased significantly in recent years. This review presents tools presented in major 

conferences, including the American Society of Heating Refrigeration and Airconditioning Engineers (ASHRAE) national meetings, the International Conference on 

Enhanced Building Operations (ICEBO), and the National Conference on Building 

Commissioning (NCBC). The international commissioning tools/prototypes presented in 

the review are limited to those developed and tested as part of the International Energy

Agency's ECBCS Annex 40, focused on Commissioning of Building HVAC Systems for 

Improving Energy Performance. These tools included guidelines on commissioning 

procedures as well as prototype software that can be implemented in stand-alone tools or 

embedded in building energy management systems. Details on the annex tools described 

in this review are available in the final report and CD, downloadable from 

www.commissioning-hvac.org. 

In most cases, tool development progresses from lab-tests to field-tests then to production 

prototypes before being considered commercially-available. The automated 

commissioning tools listed below are presented in a category best describing their most 

recent development status. In total, three commercially-available commissioning tools 

and six prototypes were identified. The collection of tools described in this section 

represent current and emerging US and international automated commissioning tools, 

beginning with commercially-available products. A general description is provided along 

with the tool name, information on the funding organization(s) and the relevant 

 61 
publication reference. Commercially-available tools and prototype tools are designated 

below as C# and P#, respectively. 

Commercially-Available Automated Commissioning Tools 

C1. ENFORMA, commercialized in 1996, Architectural Energy Corporation, USA 

(with Electric Power Research Institute cost-share) (Frey 1999). ENFORMA is 

portable diagnostic solutions software that can be used in the commissioning of new 

or existing buildings. It is useful for developing commissioning plans, collecting 

data. It analyzes data using mode-specific performance rules and guidelines to 

detect and report faults. It also generates visualization aids to help users identify 

installation and operation problems and optimize operation of building systems. An 

add-on to this tool is MicroDataNet Systems, a wireless data acquisition product 

line that provides internet access to equipment data by means of spread spectrum

wireless technology. Other features include automated metrics. 

C2. Performance and Continuous Recommissioning Analysis Tool (PACRAT), 

commercialized in 1998, Facility Dynamics Engineering, USA (Santos and 

Brightbill 2003). PACRAT is designed to review recorded meter data and other 

operational data and perform diagnostic checks for system problems, poor 

performance and energy waste using a combination of detection methods, including 

visualization tools, historical process data, expert rules, and a cost analysis. It 

analyzes trend data to provide an ongoing baseline of building performance. 

Diagnosis is based primarily on expert rules. 

C3. Virtual Mechanic, commercialized in 1996, Field Diagnostics Services, USA 

(http://www.fielddiagnostics.com). The Virtual Mechanic was developed as an 

embedded diagnostic tool for refrigeration equipment. It provides datalogging, 

calculates air conditioning performance indices that are the basis of the fault 

detection algorithm, provides alarm notification, and generates reports and analyses. 

Automated Commissioning Tool Prototypes 

P1. Whole Building Diagnostician (WBD), pilot projects since 1998, Pacific 

Northwest National Laboratory, (US Department of Energy funding), USA 

(Katipamula et al. 2004). The WBD is a production-prototype software package for 

automated diagnostics in buildings. It has two main diagnostic modules that use 

sensor data from a building’s direct digital control (DDC) system to identify 

problems and suggest solutions. The outdoor air economizer (OAE) diagnostician 

uses an expert rule set to detect and diagnose air-handling unit faults relate to 

outside air control and economizer operation. Multiple field-tests were completed. 

P2. Diagnostic Agent for Building Owners (DABO), pilot projects since 2001, 

Natural Resources Canada (Choiniere et al. 2003). DABO, the Diagnostic Agent 

for Building Operators, is an EMCS-assisted commissioning tool using expert 

knowledge to identify these faults through the use of a hybrid knowledge-based 

system composed of an Expert System and a Case-Based Reasoning module. 

DABO is not yet available on the market but is being implemented in 15 

demonstration sites in Canada. 

 62 
P3. CITE-AHU (Commissioning the Installation and Technical Equipment-AirHandling Unit), prototype and pilot project since 2003, National Institute of 

Standards and Technology, (US Department of Energy funding), French Center for 

Building Sciences (CSTB), France (Castro and Vaezi-Nejad 2005). CITE-AHU is 

an automated commissioning tool for air-handling units. It uses a library of test 

scenarios to automatically run functional performance tests and uses a rule-based 

approach to detect and diagnose faults in both constant and variable air volume

systems. It also has the capability of automatically generating test reports. The 

prototype of this tool was developed in 2003 and it has been field tested in several 

four locations. 

P4. Semi-automated Functional Test Data Analysis Tool, lab-tested prototype, 

Lawrence Berkeley National Laboratory, USA (STAC funding: California Energy 

Commission and US Department of Energy) (Xu et al. 2005). The tool is designed 

to automate the analysis of functional tests that have been performed manually. A 

set of functional test procedures has been designed to test the following air-handling 

unit components: mixing box, heating coil, cooling coil, supply fan, and return fan. 

Field measurements are entered manually into the tool by the user, avoiding the 

need for communication with the control system. The analysis tool uses simple 

mathematical models to define correct operation and uses expert linguistic rules and 

fuzzy inferencing to perform fault diagnosis. Field tests of this tool are due to take 

place in late 2005. 

P5. WebE, prototype in 2003, VTT Building and Transport, Finland (Paiho et al. 2003). 

WebE, a tool developed by the Finnish team of IEA Annex 40, is a collection of 

modules for building energy management. Of particular interest in the continuous 

commissioning application is the ability to show likely causes for system

malfunctions detected by the tool using sampling and data analysis 

P6. Control Logic Tracer, Kajima Corporation, Japan (Shioya et al. 2003). The

Control Logic Tracer is a tool developed by a Japanese research group as part of 

IEA Annex 40. It is designed to check the operation of the control logic by 

providing the designer or building operator with a means to visualize the operation 

of the control algorithm. The program reads operational data in extensible Markup 

Language (XML) format and displays the control sequence as a diagram that 

actively indicates the control path during operation. 

Table 3.1 presents the tool name and the type of commissioning, lists the type of system 

targeted, lists the end-users that the tool is designed for, provides a description of the 

tools use, and gives an indication of the level of automation provided. Although the tools 

described in Table 3.1 are in varied stages of development, all have been lab tested and or 

field-tested. The list of prototypes is not comprehensive as there are many projects 

underway without published references but this list is representative of ongoing research. 

 63 
Table 3.1. Description of Automated Commissioning Tools 

Tool, Cx Type 

System Main End-users Description

ENFORMA Initial Cx Retro-Cx

HVAC-C

hiller, 

 A

H

U diagnostics, 

 Terminal Unit Controlsperformance Lighting performance 

Facility Operators Facility Managers Energy Service Co

mpany 

Energy Managers 

ENFORMA® Portable Diagnostic Solutions software 

 creates a metering plan, determines t

he sensors nee

ded

 time-sy

nchronized building system data is o

btained via t

he

MicroDataLogger or using the EMCS 

 software manages, calculates and filters the data, then

generates 

diagnostic plots to assist i

n

diagnosi

n

g facility an

d syste

m

problems

MicroDataNet Systems provides wireless transmission of equipmentdata to the i

nternet 

PACRAT 

Continuous 

Re-Cx

Utility meterin

g

Chiller diagnostics Hydronic Distri

b

ution 

AHU diag

nostics 

Facility Operators Facility Managers Energy Service Co

mpany 

Facility Plan

ners/ 

Designers Energy Managers 

Performance and Continuous 

Recom

missioning 

A

nal

ysis Toolutilizin

g recorded system o

peratio

nal data to improve facility 

operations a

n

d planni

n

g. 

 Diagnoses system problems and poor performance and i

dentifies 

energy wastes 

 Doc

u

ments system operational parameters such as loads, e

nergy

use, and provides means to monitor a

nd verify energy uses

 Summarizes and formats the data for effective vis

ualization 

Virtual Mechanic 

O

ngoing C

x

Refrigeration c

ycles Field Engineers A portable tool for data acquisition, fault detection a

n

d diagnostics 

for roof-to

p u

nits 

 monitors refrigeration cycles and interprets 

data in applicatio

ns

including air con

ditioning a

nd refrigeration

 generates alarms, pages designated equipment and can shut off

equipment

Automated: Semi-automated: Manual: 
Tool, 

Cx Type 

System Main End-users Description

WBD 

Ongoing Cx

Electric and gas monitoring Chiller monitoringEconomizer diagnostics 

Building Operators Whole Building Diagnostician- Dual module software tool using

DDC sensors to analyze building performance 

 OAE module- automated tool for continuous analysis of

economizers 

 WBE module- automated tool for continuous analysis of whole 

building or central plant energy consumption 

 Provides potential solutions to users 

DABO 

Ongoing Cx

AHU, VAVdiagnostics 

Building OperatorEnergy Service Company Maintenance Company 

Diagnostic Agent for Building Operators- Commissioning module 

 Tool continuously monitors the building control data and stores 

it in a database 

 Database serves as a server for reasoning algorithms that 

analyze data, perform automated tests of components orsystems, identify and diagnose faults, and evaluate potential energy efficiency improvements 

 Generates reports documenting results for end-users. 

CITE-AHU Re-Cx

AHU diagnostics Building Operator

Energy Service Company Maintenance Company 

Commissioning the Installation and Technical Equipment-AHU--Automated commissioning software for air-handling units

 Applicable to both constant and variable air volume systems  Execute scenario software enables the development/storage of

automated test library 

 Data analysis can be run in real-time or as a batch process  Faults detected and list of probable causes presented to user

along with relevant sensor and control signal plots 

Automated: Semi-automated: Manual: 
Tool, 

Cx Type 

System Main End-users Description

LBNL 

Functional

Perf.Tests 

AHU diagnostics  Mixing box, heating/cooling coils, supply/returnfans 

BuildingOwner/Operator Commissioningproviders 

Semi-Automated Functional Test Data Analysis Tool

 Manual entry of measurements via graphical user interface  Analyze data, compare measured vs. expected performance  Assess system performance, identifying likely causes of failure 

WebE Ongoing Cx

Energy consumption monitoring 

Building OperatorBuilding Owner EMCS Supplier

Web-based tool to facilitate Building Energy Management

 Building information from National Building Register  WebEtana - Energy consumption estimator  WebKulu - Energy Consumption monitoring module  Fault diagnostics and commissioning of building energy systems 

based on deviation of consumption from estimates and/orhistorical data

Control Logic 

Functional 

Perf.Tests

Control system Building Operator

EMCS Installer EMCS Designer 

Graphical tool to check the sequence of operation 

 Provides transparency of HVAC control logic  Allows user to visualize control sequence over time  Diagnose failures traceable to system control Provides useful information to correct operation or control

failures 

Automated: Semi-automated: Manual: 
I. Related Research 

Although the number of automated commissioning tools is small, there are continued 

efforts underway to facilitate the implementation of the commissioning process and the 

development of new tools. Many researchers, including companies, are working to 

develop various tools to advance the market availability of automated commissioning 

tools. In addition to developing new technologies, this work includes the development of 

enabling tools and studies to advance the capabilities of diagnostic methods. The five 

tools described below are sample of resources designed to assist commissioning agents, 

planners and automated commissioning tool developers. 

Enabling Tools

The first two tools presented in this section are more recent examples of a class of data 

management tools that enable easy data collection and improves data visualization 

capabilities. Two early examples of data management tools include the Enterprise 

Energy Management Suite (EEM Suite) and the Information and Monitoring Diagnostic 

System (IMDS). The EEM Suite, commercialized in 1998 (Itron, Inc., USA, 

http://www.itron.com), provides continuous display and manipulation capabilities for 

utility and EMCS data. It is a tool designed to aid energy managers in the visualization of

system performance and is an example of a tool often referred to as Energy Information 

Systems. The IMDS prototype was first implemented in 1998 by Supersymmetry, 

LBNL, EN-Wise, C.Shockman, A. Sebald (CIEE, US Department of Energy Funding) 

(Piette et al. 2000). The IMDS was a high-quality metering and monitoring system

intended for a sophisticated building operator or commissioning agent and was designed 

to demonstrate the benefits. 

E.1 EnergyWitness, commercialized in 2005, Interval Data Systems, USA 

(http://www.intdatsys.com). It collects and archives data from multiple sources, 

including the building automation system (BAS) to visualize, detect, and diagnose 

faults. It is an on-site application.). This tool is one example of a class of data 

management tools that enable easy data collection and improves data visualization 

capabilities. Two early examples of data management tools include the Enterprise 

Energy Management Suite (EEM Suite) and the Information and Monitoring 

Diagnostic System (IMDS). The EEM Suite, commercialized in 1998 (Itron, Inc., 

USA, http://www.itron.com), provides continuous display and manipulation 

capabilities for utility and EMCS data. It is a tool designed to aid energy managers 

in the visualization of system performance and is an example of a tool often 

referred to as Energy Information Systems. The IMDS prototype was first 

implemented in 1998 by Supersymmetry, LBNL, EN-Wise, C.Shockman, A. 

Sebald (CIEE, US Department of Energy Funding) (Piette et al. 2000). The IMDS 

was a high-quality metering and monitoring system intended for a sophisticated 

building operator or commissioning agent and was designed to demonstrate the 

benefits. Energy Expert, commercialized in 2005, NorthWrite, USA 

(http://www.energyworksite.com). This tool enables near “real-time” response to 

unusual energy usage. It uses direct meter readings through an on-site gateway as 

input data. It provides data visualization, automated and manual fault detection and 

supports manual fault diagnosis. 

67
E.2 Cx Assistant, Energy Design Resources (Pacific Gas & Electric Funded) (Sweek et 

al., 2004). This process tool assists the user to organize the commissioning process 

and provides tools to help define the appropriate scope for a particular project. 

E.3 MQC Matrix, Kyoto University, Tonets Co., Japan (Nakahara and Kamitani, 

2002). The MQC Matrix is also designed to facilitate the organization of the 

commissioning process. It structures each phase and step and provides drill down 

capabilities to store and retrieve relevant information, accessible by all parties.

E.4 Commissioning Test Shell, National Institute of Standards & Technology, (Castro 

et al., 2003). The test shell was developed as part of IEA Annex 40 to provide a 

means to test multiple commissioning tools using the same data. The tool interfaces 

to the Virtual Cybernetic Building Testbed, a building simulation/emulation. 

E.5 CACEA, ComIT, Facility Dynamics Engineering, USA. CACEA is a design and 

cost management tool that can work in conjunction with ComIT. ComIT 

establishes a communications link to commissioning providers to enable the sharing 

of files and structuring of the commissioning process.

E.6 Commissioning Test Protocol Library (CTPL), Pacific Gas & Electric (Gillespie 

et al., 2001). The CTPL is a database of component, equipment, and sub-system

level protocols. Over 600 protocols were reviewed and rated. The library includes a 

subset, providing: the test name, the conditions under which the test is to be 

performed, test duration, data to be gathered, method and location of measurements 

required, instrumentation and data acquisition requirements including measurement 

tolerance, results to be obtained including analysis calculations if required, specific 

measurable acceptance criteria and any notes to the user. 

E.7 CMU-BC prototype, Carnegie Mellon University (Turkaslan-Bulbul et al.2005, 

Wang et al. 2004). The CMU-BC prototype is a system that models the processes 

and products of building commissioning. The first version of the CMU-BC 

prototype is a comprehensive implementation dealing with the AHU. It is an indepth (comprehensive sets of attribute-value pairs), and evolutionary prototype (i.e., 

it can be expanded and become more inclusive). This prototype has been used 

effectively to undertake data translation between different automated applications, 

databases and product models, which lies at the heart of interoperability technology. 

E.8 Universal Translator, Pacific Energy Center/PG&E. Is a translation tool « Rosetta 

stone » that prepares performance data from multiple sources for evaluation. This 

tool is publicly available, and has approximately 200 users. 

E.9 Energy Charting and Metrics. PECI. Is an excel spreadsheet that provides flexible 

metrics and charts in a user-friendly format. It is nearing its pilot test and will be 

made publicly and freely available. 

Table 3.2 provides more details on the nine enabling tools, presenting the tool status, its 

application, and targeted end users, and a description of its features. 

68
Table 3.2. Description of Enabling Tools 

Tool/ Status Application Main End-users DescriptionEnergyWitness

Commercially 

available 2004CxAssistant/ Commercially available 2004

All system data e.g., chillers, cooling towers, boilers, distribution loops, air handlers, and buildings Initial CxOngoing Cx

Energy Service Company Facility Planners/ Designers Energy Managers 

Energy Witness has 8 modules:EWDataCollector and EWDataWarehouse- for data handling Einterface for visualizing data, monitoring systems, and performEWIssuesTracker - for problem recording, tracking and commuEWPhysicalPlant - the main diagnostic component for chillers, 

mathematical model to compute operating cost and compares it EWUtilityBilling - for cost-allocation, invoicing;EWPurchasedUtilities - for management of purchased utility biweb-distribution of web-based tool to facilitate organization of 

 Designed to provide project-specific info to design teams  Enables user to evaluate probable commissioning cost, iden

commissioning scope 

 Provides sample commissioning specifications relevant to p

Energy Expert All system datawhole building

or sub-meteredsystems 

Energy Managers Provided as a software service via Web to Building owners, Facility Operators Facility Managers 

Energy Expert enables daily tracking of energy usage, including enables a near real-time response to atypical energy usage. Perfrecorded when deviations are deemed statistically significant ancost threshold. The tool documents analysis results and also propersistence of energy savings measures. 

Cx Assistant/ Commercially available 2004

Initial Commissioning

Designers Web-based tool to facilitate organization of commissioning pro

 Designed to provide project-specific info to design teams  Enables user to evaluate probable commissioning cost, iden

commissioning scope 

Provides sample commissioning specifications relevant to proje

Standard Models of Commissioning Plans (SMCP) 

All Cx Types Commissioning

Provider, BuildingOwners

A typical description of commissioning actions all along a projeguideline to define the commissioning plan for a given project. 

MQC Matrix/Available 2005

Initial Cx Microsoft Excel-based tool to facilitate organization of commis

 Includes commissioning plan, other elements and reference

Designers 

69 
Tool/ Status Application Main End-users Description

Commissioning TestShell/ Available in2001

Tool developmentand testing

Tool Developers Communications software that facilitates data transfer from BA

database 

 Enables the comparison of multiple commissioning/diagno

same data 

CACEA, ComIT/ Commercially available in 2000

Initial Cx / general

EMCS designer, Commissioning Agent 

Controls and Commissioning Engineering Application 

 Stores knowledge base of commissioning procedures, auto

procedures and creates graphical elements with links to issuetc. 

 Enables the organization and sharing of system information 

multi-level tree using a standard configuration

ComIT- a collaborative commissioning tool that uses the internmembers

CTPL/ Developmental Release 1.3.1, completed March2003

All Cx types, various systems 

CommissioningProviders 

Commissioning Test Protocol Library, an informational databascomponents:

 Document and protocol review database  Library of 630 existing commissioning related protocols  User protocol archive  Protocol templates document

CMU-BC All Cx types, 

AHUs 

Programmers, Designers, CommissioningProviders, Facility Managers 

Supports computer based applications by providing a complete attributes of

 equipment, equipment components, and their properties  measurable values and ranges for functional inspections  measurable values and ranges for functional performance tSupports interactive encoding of Cx measurements, their value of documents and report, such as  FPT protocols, reports, and instructions Interim and final Cx reports 

70 
Tool/ Status Application Main End-users Description

Universal Translator Data 

Acquisition andAnalysis 

Anyone managing building performance data

A translation tool that prepares performance data from multiple 

 Data is collected and imported into the software tool Corrections can be applied to data if needed Data is resampled to synchronize data interval  Graphing tool provides visualization Data can then be used to identify performance issues. 

Energy Charting and Metrics 

Tracking building performance/Data analysis 

Building Operators RetroCx providers 

A stand-alone batch tool linked to Microsoft Excel. The tool astracking and the inter-relationships between data points. 

71 
Key References for Diagnostic Tools 

In addition to the enabling tools, there are a large number of diagnostic tools with varied 

levels of automation. Much of this research has formed the basis for the analysis 

component of emerging commissioning tools, specifically tools designed for functional 

performance testing and ongoing commissioning. Using the operational data, various 

methods can be implemented to determine whether the system(s) are operating as 

intended. 

Research relating to diagnostic tools has been performed both in the US and 

internationally. Katipamula and Brambley (2005a, 2005b) present reviews of methods 

for fault detection, diagnostics and prognostics for chillers and air-handling units. An 

earlier review by Friedman et al. (2001) also included a review of manual diagnostic 

tools. International efforts include products resulting from International Energy Agency 

ECBCS Annex 34, focused on Computer-aided Evaluation of HVAC System

Performance, completed in 2000. 

In addition to these, it is worthwhile to mention an ongoing project, the “Development, 

Implementation and Deployment of Automated Fault Detection and Diagnostics for 

Vapor Compression Equipment, which was started in June 2005 with a two year duration. 

The project team of Purdue University, Field Diagnostic Services, Ben Franklin 

Technology Partners, and Honeywell Inc. are working to develop tools that could be 

applied to several types of vapor compression equipment. It is anticipated that these 

development could automate portions of the commissioning/recommissioning process for 

certain types of equipment. 

A more exhaustive list of references to methodologies and related tools (i.e., fault 

detection and diagnostic tools) is included in the bibliography. 

The basic methods used for detecting faults in buildings include: 

• Visualization tools for manual detection 

• Models derived from design information and manufacturers’ data 

3⁄4 First-principles models 

3⁄4 Semi-empirical models

3⁄4 Empirical models 

• Models derived from process history 

3⁄4 Reference comparison plots 

3⁄4 Statistics reports 

3⁄4 Benchmarking 

3⁄4 Semi-empirical models

3⁄4 Empirical models 

• Performance metrics (energy use, efficiency) 

• Guidelines for correct operation, cost estimates 

• Expert rules 

• Modeled baselines 

3⁄4 Quantitative methods 

 72 
3⁄4 Qualitative methods 

3⁄4 Physics-based methods 

IV. Market Penetration 

The market penetration of commissioning is gradually increasing, strengthened by the 

greater involvement of key players in the buildings industry, including: building 

operators, owners, managers, consultants, and builders. It is evident that as 

commissioning is increasingly applied, the need for qualified technical experts will 

increase. As stated in the review of cost-benefit methodologies, it is important to 

examine the role that automated tools have on improving the cost-benefit ratio. It is 

anticipated that the use of automated or semi-automated commissioning tools will enable 

more people to perform these functions in a more efficient manner than at present. 

Overall, there is good but limited anecdotal evidence showing the value of automated 

commissioning tools but there are few tools available and current and emerging tools 

need to be more robust to increase the potential applications. 

Barriers 

The main barriers to commissioning are that: 

1. Commissioning is a manual, time-consuming task 

2. Commissioning is seen as a cost instead of as an investment 

3. Documented commissioning methods are currently limited to conventional 

HVAC systems 

4. There is a lack of technical experts and tools for field optimization, 

commissioning, and data visualization 

5. Information is lost between design and commissioning. 

6. The lack of standardized data handling schemes increase project costs 

7. Market planning during tool development is inadequate 

8. Measures are not taken to ensure persistence of benefits 

Potential Benefits 

Because the current commissioning process is both time-consuming and expensive and 

there is a need to commission both new and existing buildings for improved energy 

performance, the value of these potential benefits is substantial. Many of the tools and 

tool prototypes described in this paper provide means to facilitate aspects of the 

commissioning process for particular HVAC systems; however, there is also a great 

potential for improving these tools. Some of the potential benefits of automation listed by 

Brambley et al. (1999) are shown italicized in Table 3.3. These potential benefits still 

exist and advances have been made. The table summarizes the current state of automated 

tools for commissioning, including limitations that could be addressed by further R&D. 

 73 
Table 3.3. Potential Benefits of Automated Tools and Current State of 

Development

Potential Benefits Current State of Automated Tools 

Speed up the process of preparing a 

commissioning plan, ensure compliance with 

standards/guidelines, and help ensure 

consistency across projects. 

Several enabling tools provide templates, guidelines and 

file-sharing features that can begin to address this goal. 

The current limitation is in the types of building systems 

covered. 

Speed up the process of detecting and 

diagnosing problems with the operation of

heating, ventilating, and air conditioning

equipment and systems 

Tools exist for a variety of conventional HVAC systems, 

however, due to the near-infinite variations in system

designs and control algorithms, time and expertise needed

for tool configuration is a limiting factor. 

Improve quality of data handling by

eliminating the possibility of introducing

errors through manual data entry 

Access to EMCS data is a valuable tool. Many

proprietary formats still exist which impede data access, 

particularly for existing buildings. 

Disseminate expert knowledge by embedding 

it in software tools, Ensure consistency in 

fault detection and diagnosis across 

buildings, projects and different

commissioning agents 

The embedded knowledge, combined with the increased

transparency of system operation through improved

visualization tools can greatly empower building

operators 

Disseminate expert knowledge by embedding 

in manufacturer’s equipment.

Prototype embedded implementations of diagnostic tools 

have already being tested (Schein et al. 2003), with

promising results. 

Archive data electronically for future

reference or use. 

Tools developed that reside in the EMCS or having good

data access can perform added functions such as 

automatically generate necessary documentation, provide

access to historical data to show how system use has 

progressed over time. 

References 

Key Automated Commissioning Reports 

Brambley, Michael and Srinivas Katipamula. 2003. “Automating Commissioning 

Activities: An Update with an Example,” in proceedings of the 11th National Conference 

on Building Commissioning. 

Brambley, Michael et al. 1999. “Use of Automated Tools for Building Commissioning,” 

in proceedings of the 7th National Conference on Building Commissioning. 

Brooks, Martha, and Nancy Jenkins. 2003. “Overview of PIER-Funded Existing 

Building Commissioning and Diagnostics Research” in proceedings of the International 

Conference for Enhanced Building Operations. 

Castro, Natascha and Hossein Vaezi-Nejad. 2005. “CITE-AHU: An Automated 

Commissioning Tool for Air-Handling Units” in proceedings of 13th National 

Conference on Building Commissioning. 

 74 
Choiniere, Daniel and Maria Corsi. 2004. “Four Years of On-Going Commissioning in 

CTEC-Varennes Building with a BEMS Assisted Cx Tool” in proceedings of the 4th

International Conference for Enhanced Building Operations. 

Frey, D.1999. “Diagnostics Based on Short-Term Monitoring,” Proceedings of the 

Workshop on Diagnostics for Commercial Buildings: From Research to Practice, June 

16-17, 1999, Pacific Energy Center, San Francisco. 

Friedman, Hannah and Mary Ann Piette. 2001. “Comparison of Emerging Diagnostic 

Tools for Large Commercial HVAC Systems” in proceedings of 7th National Conference 

on Building Commissioning. 

Gillespie, K and Bruceri, M. 2001. “Library of Commissioning Test Protocols”, in 

proceedings of the 9th National Conference on Building Commissioning. 

Heinemeier et al. 2007.“Data Analysis and Diagnostic Tools for Commissioning: A 

Demonstration Workshop” in proceedings of the 15th National Conference on Building 

Commissioning. 

Katipamula, Srinivas, and Michael Brambley, 2005 “Methods for Fault Detection, 

Diagnostics, and Prognostics for Building Systems—A Review, Part I” ASHRAE 

HVAC&R Research Volume 11, Number 1. 

Katipamula, Srinivas, and Michael Brambley, 2005 “Methods for Fault Detection, 

Diagnostics, and Prognostics for Building Systems—A Review, Part II” ASHRAE 

HVAC&R Research Volume 11, Number 2. 

Paiho, Satu, et al. 2004, “Web-based Services for Building Energy Management-WebE,” 

proceedings of the 4th International Conference for Enhanced Building Operations.

Piette, Mary Ann et al. 2000. “Use of an Information Monitoring and Diagnostic System

to Improve Building Operations” ACEEE Summer Study. 

Santos, J. and Brightbill, 2003. “PACRAT: The Latest in Automated Diagnostics”, in 

proceedings of the 11th National Conference on Building Commissioning.

Shioya, Masaki, et al. 2004 “Study on Commissioning Process for Control Logic of 

Thermal Storage System” in proceedings of the 4th International Conference for 

Enhanced Building Operations. 

Turkaslan-Bulbul, M.T. and Akin, O., 2005. “Computational Support for Building 

Evaluation: Embedded Commissioning Model” in Proceedings of The First Conference 

on the Future of the AEC Industry Las Vegas, Nevada and in Journal of Automation in 

Construction, forthcoming. 

Visier, Jean-Christophe et al. 2005. “Commissioning Tools for Improved Energy 

Performance: Results of IEA ECBCS Annex 40” downloadable from

www.commissioning-hvac.org

 75 
Wang, H., Akinci, B., Garrett, J.H., Akin, O., Turkaslan-Bulbul, M.T. and Gursel, I., 

2004. “Towards Domain-Oriented Semi-Automated Model Matching for Supporting 

Data Exchange” in The 10th International Conference on Computing in Civil and 

Building Engineering, Weimar, Germany 

Xu et al., 2005. “A Semi-Automated Functional Test Data Analysis Tool” in proceedings 

of the 13th National Conference on Building Commissioning. 

 76 
Website References 

ENFORMA, http://www.archenergy.com/products/enforma/ 

PACRAT, CACEA, ComIT, www.facilitydynamics.com

Virtual Mechanic, http://www.fielddiagnostics.com/virtualmechanic.cfm

EnergyWitness, http://www.intdatsys.com/EnergyWitness.htm 

EEM Suite, Enterprise Energy Management www.itron.com

WBD, http://www.buildingsystemsprogram.pnl.gov/content.htm

IEA Annex 40, www.commissioning-hvac.org 

Additional Diagnostics Reports 

Bianchi, M. et al. 2003. “Evaluating and Documenting the Performance of Unitary 

HVAC Equipment with the Honeywell HVAC Service Assistant” in proceedings of 12th 

National Conference on Building Commissioning. 

Bushby, Steven, et al. 2003. “A Test Shell for Developing Automated Commissioning 

Tools for BACnet Systems” in proceedings of 11th National Conference on Building 

Commissioning. 

IEA EBCBS Annex 34 Final Report, “Demonstrating Automated Fault Detection and 

Diagnosis Methods in Real Buildings” Proceedings of VTT Symposium: 217. 

Katipamula, Srinivas, Michael Brambley, Nathan Bauman, and Robert Pratt. 2004. 

“Enhancing Building Operations through Automated Diagnostics: Field Test Results” in 

proceedings of the International Conference for Enhanced Building Operations. 

Rossi, T. 2004. “HVAC Diagnostics” in proceedings of 12th National Conference on 

Building Commissioning. 

Schein, J., and House, J.M. 2003. Application of Control Charts for Detecting Faults in 

Variable-Air-Volume Boxes. ASHRAE Transactions Vol. 109, Part 2. 

Schein, J. and Bushby, S.T. 2005. A Simulation Study of a Hierarchical, Rule-Based 

Method for System-Level Fault Detection and Diagnostics in HVAC Systems. NISTIR 

7216. National Institute of Standards and Technology. 

Schein, J., Bushby, S.T., and House, J.M. 2003. Results from Laboratory Testing of 

Embedded Air Handling Unit and Variable Air Volume Box Diagnostic Tools. NISTIR 

7036. National Institute of Standards and Technology. 

 77 
Chapter 4: Review of Needs and Challenges in 

Commissioning Zero Energy Buildings 

I. Introduction 

This section reviews the particular issues that apply to the commissioning of aggressively 

low energy buildings and that can be expected to apply to buildings that approach zero 

energy consumption. A literature review of commissioning and operational experience 

with existing buildings that were designed to have low energy consumption has been 

undertaken. Key findings from a number of case studies are reported and generic 

conclusions with implications for R&D are presented. 

Based on both recent low energy designs and current thinking, buildings that approach

zero energy consumption can be expected to have most or all of the following 

characteristics: 

• Dramatically reduced electrical and thermal loads 

• Remaining loads met by solar (photovoltaics and solar-thermal) 

• Very efficient components and subsystems

• Unconventional systems 

• Close integration of systems: 

3⁄4 HVAC 

3⁄4 Lighting 

3⁄4 Daylighting and facade 

3⁄4 On-site generation 

• (Near-)optimal control 

Many of these characteristics are found in recently constructed ‘green’ buildings, 

especially those designed to achieve Gold and Platinum Leadership in Energy and 

Environmental Design (LEED) ratings. As regards commissioning, ‘Fundamental 

Building Systems Commissioning” is required for all LEED-rated buildings. An 

additional LEED point is available for ‘Additional Commissioning’, which makes it more 

likely that a more comprehensive commissioning process will have been followed for 

Gold and Platinum buildings. Since these buildings typically have more innovative 

systems in order to meet their more aggressive energy consumption targets, they are the 

subjects of choice for the study of issues relating to the commissioning of very low 

energy buildings. Unfortunately, there is relatively little detailed information available 

that is relevant to the commissioning of these buildings. For example, almost all of the 

case studies on the US Green Building Council website13 make no mention of 

commissioning, even for Gold and Platinum projects. This situation may improve over 

time but it does suggest a need to be proactive in obtaining information on design and 

operational problems in buildings that are likely to be seen as successful. 

 

13 Linked from http://www.usgbc.org/LEED/Project/project_list.asp

 78 
II. Literature Review 

The publications reviewed and described here fall into two categories: 

• Case studies of individual buildings 

• Generic lessons learned from commissioning ‘green’ buildings 

Case Studies 

The most comprehensive sources of information found are the individual reports on the 

six High Performance Buildings studied by the National Renewable Energy Laboratory 

(NREL) and also described in Torcellini et al. (2004). The ‘lessons learned’ sections of 

the reports typically include descriptions of: 

• Design problems that could have been addressed in the design review phase of a 

formal commissioning process 

• Construction, equipment and control problems that could have been identified by 

functional testing 

• Operational problems that would have required some form of performance 

monitoring and/or post-occupancy evaluation for them to have been identified had 

NREL not performed the case study. 

The first six items below summarize the commissioning-related issues identified in the 

NREL High Performance Buildings reports. The seventh item summarizes the 

commissioning-related issues identified in a case study on a showcase new office 

building in Berlin, Germany. 

1. BigHorn Home Improvement Center (Deru et al. 2005a)

• No 3rd party commissioning – considered too expensive 

• Lack of documentation during design and construction led to disputes about intended 

design and responsibilities 

• Limitations of DOE-2 restricted the design 

• Daylight performance was poorer than anticipated – actual glazing area less than 

designed, bug screens reduced transmission, dark overhangs. 

• PV and inverters not designed as an integrated system - also needs automated 

monitoring. Poor payback as peak demand is from lighting, i.e. at night – worth 

installing batteries? 

• Transpired solar collectors for ventilation preheat ineffective as warehouse doors 

open all the time. 

• Poor design of hydronic systems (radiant floor and snowmelt) 

• Poor light sensor placement 

• Poor integration of lighting controls and EMCS (switches in series) 

• Natural ventilation controls inadequate initially – need to take account of current and 

future outside temperatures. 

• Sequence of operations for boiler and pump had to be reprogrammed 

• Actuators of clerestory windows had to be rewired to work with EMCS 

• General need for better performance monitoring – installed EMCS not adequate. 

 79 
• Programming the EMCS an ongoing process to fine-tune the performance of the 

building 

• Performance benefited from post-occupancy fine-tuning of system operations. This 

involves constant effort, which requires motivated and trained staff – absent in most 

buildings. 

2. Cambria Office Building (Deru et al. 2005b)

• LEED Gold

• The design-build process restricts integrated design by creating a dynamic where 

systems are designed in series 

• Daylighting performance and artificial lighting performance both poorer than 

expected 

• Photovoltaic inverter has 18 % losses – disconnect when not in use or (preferably) 

replace with more efficient unit. 

• Photovoltaic system needs automated performance monitoring – manual monitoring 

requires unsustainable effort. 

• Underfloor air distribution (UFAD) system has slow response time; temperature 

setbacks should be small and have long start-up times. 

• (Conventional) commissioning is important but does not guarantee that the occupied 

building will operate efficiently. Monitoring building end use energy consumption 

provides valuable feedback to help maintain efficient performance of systems. Two 

main items missed during commissioning were (that) the west energy recovery 

ventilator ran continuously and the HVAC fans ran continuously. These items might 

have been difficult to detect without looking at the end-use data. The end-use data 

also allowed the timing of the lighting circuits and HVAC controls to be changed to 

save energy. 

• Performance monitoring systems need to be very robust and be actively maintained 

3. Chesapeake Bay Foundation’s Philip Merrill Environmental Center (Griffith et al.

2005)

• LEED Platinum

• Natural ventilation design used assumed rather than measured wind direction. Better 

to design for multiple wind directions and/or stack unless wind direction reliable 

• Dark exposed beams and ducts reduce daylight penetration 

• Photovoltaic panels shaded by exterior structure. 

• Economizer omitted from design 

• Desiccant wheel installed but not used 

• Demand charges not taken into account when staging heat pumps 

• ‘Usual’ commissioning activities only relate to design basis and don’t accommodate 

changes during occupancy. Need for post-occupancy monitoring, leading to 

refinement of control strategies etc. 

 80 
4. NREL Thermal Test Facility (Torcellini et al. 2005)

• Unique design philosophy and nonstandard equipment require special emphasis on 

control system design. 

• Vigilant field inspection of energy features needed during construction 

• Funds were available for post-commissioning tuning and alterations, which resulted 

in performance improvements that would not otherwise have been obtained. 

5. The Adam Joseph Lewis Center for Environmental Studies, Oberlin College (Pless

and Torcellini 2004)

• Heating-dominated building. 

• Electric boilers used initially to preheat water for incorrectly specified ground source 

heat pumps – heat pumps were then upgraded to work over an extended temperature 

range to reduce use of electric resistance heating. Careful staging needed for 

remaining electric preheat. 

• Atrium has poor performance, low solar heat gain coefficient 

• Photovoltaic inverter problems 

• Photovoltaic isolation transformer losses – replace with more efficient transformer 

• Lighting control improvements identified and implemented 

• Controls designer/contractor didn’t understand the building 

• Users didn’t fully understand the control system

• Detailed monitoring is needed to fully evaluate the building and to identify additional 

areas of energy savings, a level of monitoring beyond the scope of typical 

commissioning projects. 

6. Zion National Park Visitors Center (Torcellini et al. 2005)

• Passive direct evaporative cooling, natural ventilation; direct gain and noncirculating 

Trombe wall passive solar heating 

• Differences between climate at site and weather file resulted in undersizing of cooling 

capacity 

• Peak electric demand occurs on winter mornings: lights + space heating + water 

heating – stagger start-up of different loads with demand-limiting controls. This was 

difficult to implement, failed when controls reset after power outage, causing high 

peak demand. Demand-limiting controls need to be robust as one failure triggers high 

demand charge. 

• No formal commissioning. Informal commissioning by NREL personnel on site led 

to multiple problems being found and fixed; probably would not have happened 

otherwise. 

• Continuous performance monitoring allowed NREL to identify further problems and 

significantly reduce energy use 

7. EnergieForum Berlin (Plesser 2005)

• Office building in Berlin, Germany 

 81 
• 100 kW·h·m-2

·y

-1 (31 kBTU·ft-2

·y

-1) target for whole building primary energy use 

achieved, even with some missing control schedules and heat recovery problems. 

• Calibration of the various systems for building heating and cooling proved to be 

problematic. 

• Based on measurements and thermal simulations in terms of building monitoring 

(calibrated simulations?), proposed solutions including new control strategies were 

developed and implemented. A reduction of 33 % in the energy use during the 

second heating season shows the positive effects of the energy and comfort 

monitoring and the cooperation with the management of the facility. 

• Energy designers should not only support the design, planning and construction of a 

new building but also evaluate the commissioning process and the first years of 

operation. 

Though not explicitly a case study, PECI’s retrocommissioning of the Intelligent 

Workplace Laboratory at Carnegie Mellon University, identifies a number of issues that 

relate to the commissioning of low energy buildings (Sellers 2002): 

• Detailing problems encountered with unfamiliar technology – could have been caught 

by a design review of the shop drawings 

• Poor comfort control 

• Poor make-up air system performance and reliability 

• System integration problems

• Problems with coordination of control of multiple components/systems: natural 

ventilation, evaporative cooling, demand-controlled ventilation, radiant cooling ... 

Generic Lessons Learned 

Berning and Grunenwald (2004) describe several problems that they encountered in the 

commissioning of a number of LEED-certified buildings: 

• Design intent and basis of design not developed well by the design team and owner. 

• Data documenting the completion of the LEED requirements is not assembled 

appropriately or in a timely manner by the design team at the end of the design phase 

• Contractors don’t submit reports – or are reluctant to do so – documenting LEED 

compliance. 

• The project manager’s ability to enforce the requirements is often reduced by the 

contractor’s lack of understanding that this needs to be managed. 

Ring and Ingwalson (2004) also describe recurrent problems encountered in 

commissioning ‘green’ buildings: 

• LEED projects, especially Gold and Platinum, often include non-standard 

mechanical, electrical and plumbing systems, significantly increasing the effort 

required for commissioning. 

• The commitment of the Owner is the key to success – the commissioning process and 

the problems it reveals can be disregarded if the owner is disengaged or unsupportive 

 82 
• Unmanageable complexity should be avoided at all costs; innovative systems need to 

be understood by all concerned, especially O&M staff – a particular problem in small 

buildings 

• The sequence of operation is often ambiguous – poor information provided to 

controls contractor who is likely unfamiliar with the systems if they are innovative. 

Poorly defined or documented sequences of operation make it difficult to develop 

functional tests. 

• Daylight dimming systems require significant coordination. Most lighting designers, 

electrical contractors and building operators do not have enough experience with 

dimming systems to appreciate the issues related to sensor placement, zoning and 

circuiting. 

• Underfloor air delivery plenums are difficult to seal for supply air control, so difficult 

to get good air distribution. Difficult to control supply plenum pressure in VAV 

systems 

III. Discussion 

A number of recurrent themes are evident in the ten publications cited above. 

Design Problems 

Designers, even when they have been selected for high visibility sustainable design 

projects, are often not sufficiently familiar with the even moderately unconventional 

strategies and systems that they employ – e.g., daylighting, ground source heat pumps, 

photovoltaics, evaporative cooling and natural ventilation. The design review element of 

the commissioning process has the potential to trap some of the problems that arise as a 

result of the limitations of the design team, but is not a panacea in this regard. Simulation 

has a potentially beneficial role to play in that provides a means of representing the 

expected performance of the building under different operating conditions. Properly 

applied, it can identify mistakes during design and can also be used to represent expected 

performance during commissioning. Significant differences between predicted and 

measured performance can then be investigated and associated with construction 

deficiencies, equipment malfunction, design errors, incorrect simulation inputs or 

inadequate models. 

Another generic problem is that designs are often too complex for the skill level of the 

contractors and/or whoever will operate the building. This is a particular problem for 

small, isolated (i.e., not on a campus), buildings, since they rarely have a dedicated 

operator who could cope with the unique features of the building. Simpler, more robust, 

systems often perform better in practice than more complex systems that are more 

efficient on paper. This problem is a particular challenge for very low energy buildings; 

it is not yet clear to what extent commissioning, performance monitoring and automated 

diagnostics can ameliorate this problem.

Commissioning Methods Lacking for Innovative Buildings 

Formal, third-party, commissioning is frequently not performed because it is seen as too 

expensive and the benefits are not well understood. Commissioning is thought to be 

appropriate for larger buildings, where economies of scale can help to reduce the cost on 

 83 
a floor area basis. However, many of the recent buildings designed to have very low 

energy consumption, and all of the US examples that have been studied in detail, are 

envelope-dominated, small commercial or educational buildings. As a result, daylighting 

and heating are more important than they would be in larger buildings; for most of the 

buildings, daylighting plays the most important role in the design strategy. Most of the 

commissioning currently performed in the United States is restricted to conventional 

HVAC systems in offices, schools and ‘high tech’ buildings. As a result, ‘conventional’ 

commissioning is often inadequate to detect problems in innovative buildings. 

Functional test procedures are not well established for unconventional systems and are 

undefined for systems that interact at the whole building level. As a result, problems are 

usually detected by monitoring routine operation, even when commissioning ostensibly 

was performed. 

Operational Problems 

Operational problems were detected and remedied, and energy consumption reduced, as a 

result of performance monitoring by outside research organizations, e.g., NREL. It 

would be instructive to perform case studies in which external intervention was explicitly 

prohibited in the first phase of the study, so as to determine the performance that occurs 

without outside intervention. 

Controls Problems 

Controls are a frequent source of problems. Many designer teams leave the design of the 

control strategy to the controls contractor. With few exceptions, controls contractors 

have little or no knowledge or experience of innovative or unconventional mechanical 

systems, natural ventilation, dimming controls, façade controls, photovoltaics or system 

integration. Commissioning can at least serve to identify problems whose nature might 

otherwise not be understood, which is potentially a first step to these problems being 

addressed. 

IV. Conclusions and Recommendations 

The commissioning of innovative buildings presents a number of challenges, particularly 

if achieving the expected energy performance is a key objective: 

• Conventional new building test procedures typically do not explicitly address energy 

consumption at the system or whole building level. 

• Conventional commissioning procedures typically do not address peak demand. 

• Commissioning procedures for innovative systems are not available in libraries or 

guides. 

• Innovative buildings are, almost by definition, ‘one-of-a-kind’ and require functional 

test procedures that are customized to the design of that unique building. 

 R&D needs to address these challenges include: 

• Develop methods of documenting design intent that: 

3⁄4 Extend to integration of systems at the whole building level 

3⁄4 Can be clearly understood by design team, commissioning agents and operators 

 84 
3⁄4 Include a simulation model to provide a quantitative representation of the 

expected performance of the building

• Develop design review guidelines for low-energy buildings to help catch problems 

early. 

• Develop functional test methods that adequately address innovative system operation 

and integration issues. 

• Develop functional test methods that compare expected energy performance to actual 

energy performance during commissioning and diagnose causes of differences. 

One approach to the development of functional test methods that address energy 

performance in ‘one-of-a-kind’ buildings is as follows: 

• Assume that a detailed simulation model will be produced during the design phase 

• For manual testing, develop methods of using this model to generate functional test 

procedures: 

3⁄4 Identify critical operating points for system performance and interaction between 

systems 

3⁄4 Calculate expected performance of each system at these operating points 

• For automated testing, develop automated analysis procedures that compare actual 

performance to the predictions generated by a real-time version of the design model: 

3⁄4 The commissioning agent manipulates internal loads and set-points to drive the 

building to critical operating points 

3⁄4 An automated tool uses comparisons of different measured and simulated 

quantities to distinguish between different possible faults 

• ‘Energy commissioning’ requires additional sensors to monitor energy flows: 

3⁄4 Electric submetering: lights, plugs, chillers, fans, etc. 

3⁄4 Thermal flows: chilled water, hot water, air 

It is proposed to pursue this approach in the context of IEA Annex 47 using EnergyPlus 

and a research plan will be prepared and presented for discussion at the new Annex 

meeting. 

References 

Berning, M. and Grunenwald, B. “How to Achieve LEED Certification for 

Commissioning Projects”, Proceedings of the National Conference on Building 

Commissioning, PECI, New York, May, 2004 

Deru, M., Torcellini, P. and Pless, S. Energy Design and Performance Analysis of the 

BigHorn Home Improvement Center. NREL/TP-550-34930. 

http://www.nrel.gov/docs/fy05osti/34930.pdf 2005a 

Deru, M., Torcellini, P., Sheffer, M. and Lau, A. Analysis of the Design and Energy 

Performance of the Pennsylvania Department of Environmental Protection Cambria 

Office Building. NREL/TP-550-34931 http://www.nrel.gov/docs/fy05osti/34931.pdf

2005b 

 85 
Griffith, B., Deru, M., Torcellini, P. and Ellis, P. Analysis of the Energy Performance of 

the Chesapeake Bay Foundation’s Philip Merrill Environmental Center. NREL/TP-550-

34830. http://www.nrel.gov/docs/fy05osti/34830.pdf 2005 

Pless, S. and Torcellini, P. Energy Performance Evaluation of an Educational Facility: 

The Adam Joseph Lewis Center for Environmental Studies, Oberlin College, Oberlin, 

Ohio. NREL/TP-550-33180. http://www.nrel.gov/docs/fy05osti/33180.pdf 2004. 

Plesser, S. Portrait of EnergieForum Berlin. Federal Ministry of Economics and Labour. 

Berlin, Germany. 2005 

Ring, E. and Ingwalson, J. “Not that Again!: Recurring findings from the commissioning 

of green buildings”, Proceedings of the National Conference on Building Commissioning, 

PECI, New York, May, 2004. 

Sellers, D. Preliminary Findings and Recommendations from the Retro-commissioning 

Process at the Carnegie Mellon University Intelligent Workplace Laboratory - 

presentation to Carnegie Mellon University. 2002 

Torcellini, P.A, Deru, M., Griffith, B., Long, N., Pless, S., Judkoff, R. and Crawley, D.B.

“Lessons Learned from Field Evaluation of Six High-Performance Buildings”. 

Proceedings of the 2004 ACEEE Summer Study on Energy Efficiency in Buildings, Aug. 

2004 

Torcellini, P., Pless, S., Griffith, B. and Judkoff, R. Evaluation of the Energy 

Performance and Design Process of the Thermal Test Facility at the National Renewable 

Energy Laboratory. NREL/TP-550-34832. http://www.nrel.gov/docs/fy05osti/34832.pdf 

2005 

Torcellini, P., Long, N., Pless, S. and Judkoff, R. Evaluation of the Low-Energy Design 

and Energy Performance of the Zion National Park Visitors Center. NREL/TP-550-

34607. http://www.nrel.gov/docs/fy05osti/34607.pdf 2005. 

 86 
Chapter 5: New Functional Tests for use in 

Commissioning Zero and Low Energy Buildings 

1. Introduction 

Functional test procedures for low energy HVAC and envelope systems are being 

developed in the context of IEA Annex 47. Tests for the following systems were 

developed in 2005: 

• Radiant slabs for heating 

• Underfloor air distribution plenum pressure 

• Demand-controlled ventilation 

• Building pressurization 

An overview of each test is presented below. These and other test procedures developed 

in the context of the Annex will be posted on the US team website.14 

2. Radiant Slab Functional Test 

Radiant floor heating is a strategy long utilized in residential construction that is 

becoming more commonplace in commercial applications. It is an effective heating 

strategy for large open zones with high ceilings like lobbies, atriums, auditoriums, 

warehouses, light manufacturing facilities, and gymnasiums. However, it is also being 

installed in more “traditional” commercial spaces like office buildings due to the efficient 

use of energy and space, as well as reduced maintenance, associated with the system. A 

radiant floor heating system can be difficult to control due to the large thermal mass 

associated with the concrete slab and its slow response to load variations. In addition, the 

radiant floor heating system may need to interact with a conventional air handling system

designed to provide ventilation and humidity control to the space. This procedure allows 

the verification of proper operation of a radiant floor heating system, but does not directly 

address the testing of ancillary HVAC equipment that may operate in conjunction with 

the radiant floor to supply ventilation or supplemental heating/cooling to the zone. This 

procedure involves: 

• Ensuring all system verification checks are complete prior to executing radiant floor 

heating system tests 

• Verifying system responds per the design sequence of operations 

• Verifying actual system control through long-term trending 

3. Underfloor Air Distribution Plenum Pressure Test The primary objective of testing both the underfloor and return air plenums is to quantify 

the air leakage rate from each plenum, identify the leak source(s), and facilitate repair as 

necessary. Having a relatively leak-free plenum will reduce central HVAC supply fan 

energy usage, prevent system operational problems, and minimize comfort problems. 

 

14 http://www.nist.gov/annex47/

 87 
 88 

The procedure involves: 

• Ensuring all system verification checks are complete prior to executing system tests 

• Quantifying the air leakage rate from both the underfloor plenum and occupied 

space/return plenum

• Identifying the source of air leaks and repairing them as best as possible 

4. Demand-Controlled Ventilation 

A demand-controlled ventilation control strategy adjusts the quantity of outdoor 

ventilation air supplied by a central air handling unit based on the ventilation rate 

required to provide adequate indoor air quality. A significant amount of heating and 

cooling energy can be saved by supplying just enough ventilation air to satisfy zone load 

requirements. The objective of testing the demand-controlled ventilation control strategy 

is to ensure that outdoor ventilation air is adjusted as necessary to meet zone loads as they 

vary with time. The procedure involves: 

• Ensuring all system verification checks are complete prior to executing system tests 

• Verifying demand-controlled ventilation control strategy operates as intended for 

both constant and variable air volume air handling systems 

• Verifying minimum ventilation air requirements are met under varying operating 

conditions 

• Verifying that the demand-controlled ventilation control strategy interacts with the 

economizer control sequence correctly 

5. Building Pressurization 

The test is designed to help diagnose and quantify the air tightness of the whole building 

envelope by putting air-handling systems in 100 % outdoor air mode to pressurize the 

building. It is important for all buildings, but is even more critical for low energy 

buildings. The test is based on ASTM Standard E-779-99, but is less rigorous. The 

ASTM standard requires pressure measurements on every face of the building and on 

every few floors. It also advances the pressure in very small increments up and back. A 

more practical test is needed, one which could be performed more rapidly (and thus at a 

lower cost). By using commissioning grade measurements, rather than research-grade

measurements, accuracy is sacrificed in order to have a more practical test procedure. 

Preparation is essential when performing a building pressure test. The Precautions 

section and the Prerequisites section describe the most important considerations, 

including verifying safeties, selecting appropriate test conditions, finding pressure 

measurement locations, and setting up the HVAC systems for the test. After the test is 

performed, the Acceptance Criteria and the Analysis of Test Data sections help quantify 

the energy impact of leaks and provides avenues to address problems with air tightness

that are identified. 
 89 

Conclusions 

This report presents the state of the art in four areas where further research and 

development are necessary to improve the knowledge, procedures and tools that support 

the provision of effective commissioning services: cost-benefit methodologies, the 

persistence of benefits, automated commissioning tools, and the needs and challenges in 

commissioning zero energy buildings. 

The following are specific R&D recommendations in each of the four areas: 

Cost-benefit methodologies 

• Develop a standardized methodology for evaluating commissioning costs and 

benefits. 

• Create a data collection instrument that allows respondents to easily submit their 

project information and incorporates an automated or semi-automated analysis tool. 

• Fund an ongoing data collection and analysis effort. 

Persistence of benefits 

• Develop a uniform methodology for determining commissioning persistence 

• Determine the persistence of savings from a broader sample of buildings 

• Develop simple tools for tracking performance of commissioning measures 

• Develop practical methods for owners and operators to better maintain 

commissioning savings 

Automated commissioning tools

• Develop user-demanded features in automated commissioning tools, including 

automatic generation of documentation with cost information based on standard 

calculations.

• Implement information models for building systems to reduce the information loss 

and enable automated use of data as a building advances through design, construction, 

and operation.

• Extend methods and tools, for additional building systems, to speed up the process of 

preparing a commissioning plan, ensure compliance with standards/guidelines, and 

help ensure consistency across projects 

Commissioning zero energy buildings 

• Develop methods of documenting design intent and performing design reviews that 

are adapted to the specific needs of innovative and low energy buildings 

• Develop functional test methods that adequately address innovative system operation 

and integration issues. 

• Develop functional test methods that compare expected energy performance to actual 

energy performance during commissioning and diagnose causes of differences. 
 90 

• Incorporate simulation in functional testing as a means of enforcing accountability for 

energy performance between design and construction 

Commissioning has a key role to play in comprehensive quality assurance for the design, 

construction and operation of buildings. Although commissioning is a valuable means to 

ensure that a building reaches its operating potential, it has not been widely adopted. 

Overcoming the market barriers to the adoption of commissioning is becoming 

progressively more important as the number of buildings employing innovative, 

interactive systems increases. The need for commissioning is especially acute when these 

systems are installed in low or zero energy buildings. 

 
 91 

Appendix: Automated Commissioning Tool Interfaces 

ENFORMA – Architectural Energy Corporation 

Figure A.1. Diagnostic Plot shows the power draw for the roof-top unit with the properly operating 

economizer. 
 92 

Figure A.2. ENFORMA RTU plot
 93 

PACRAT- Facility Dynamics Engineering 

Figure A.3. Sample PACRAT Anomaly Report Output 

Figure A.4. Sample PACRAT Performance Graph 
 94 

Virtual Mechanic 

Figure A.5. Virtual Mechanic hardware
 95 

WBD-PNNL

Figure A.6. Sample Energy Performance Chart 

 

Figure A.7. Sample Diagnostics Report
 96 

DABO- Natural Resources Canada

Figure A.8. DABO tool

Figure A.9. DABO Configuration Interface
 97 

CITE-AHU- NIST 

Active testing of cooling coil

0

5

10

15

20

25

30

23:30 0:00 0:30 1:00 1:30 2:00 2:30 3:00 3:30 4:00 4:30 5:00 5:30 6:00 6:30

Time

Te

m

pe

rat

ur

e

0

10

20

30

40

50

60

70

80

90

100

Co

ntr

ol 

si

gn

al

Cooling coil valve

control signal

Return air

temperature

Supply air

temperatur

On/Off fan

control signal

Turn on AHU, pumps and chiller

and override the control of the

AHU

Closing request of valves,

heat recovery and

economizer.

Control signals: 0 %

Middle range operation of

the cooling coil valve.

Control signal: 30 %, 60 %

Maximum opening request

of the cooling coil valve.

Control signal: 100 %

Closing request of of the

cooling coil valve.

Control signal: 0 %

Turn off AHU, pumps and chiller.

Activate automatic mode for AHU

Automatic analysis of the

performance of the component

Automatic execution of the scenario

T

e

m

p

e

r

a

t

u

r

e

(

C)

Time of day

Figure A.10. Overview of test sequence and analysis
 98 

LBNL Tool 

Figure A.11. Project Set-up Interface 

Figure A.12. Draft GUI of the Functional test input and output submenu 
 99 

WebE 

Figure A.13. Fault Detection and Diagnostic Approach 

Figure A.14. A flowchart of the FDD/Commissioning Module 
 100 

EnergyWitness 

Figure A.15. Trending capabilities for data.

Figure A.16. Chilled Water Plant Speedometer. 

Operators can move through time (1) and see data for each system and the entire plant (2). Bypass 

flow (3), energy rates (4), and weather data (5) are shown. A tab (6) switches the view to show the 

underlying models. Gauges show total in $/hr and $/ton-hr (7). 
 101 

EnergyExpert 

Figure A.17. Energy Expert screen shots for Scorecard report (top), and tables (below). 
 102 

Control Logic Tracer 

Figure A.17. Flow chart of Control Logic Tracer operation.