NREL is a national laboratory of the U.S. Department of Energy

Office of Energy Efficiency & Renewable Energy

Operated by the Alliance for Sustainable Energy, LLC

This report is available at no cost from the National Renewable Energy

Laboratory (NREL) at www.nrel.gov/publications.

Contract No. DE-AC36-08GO28308

Portfolio-Scale Optimization of 

Customer Energy Efficiency 

Incentive and Marketing

Cooperative Research and 

Development Final Report

CRADA Number: CRD-13-535

NREL Technical Contact: Dr. Larry J. Brackney

CRADA Report

NREL/TP-5500-65919

February 2016
NOTICE

This report was prepared as an account of work sponsored by an agency of the United States government. 

Neither the United States government nor any agency thereof, nor any of their employees, makes any warranty, 

express or implied, or assumes any legal liability or responsibility for the accuracy, completeness, or usefulness of 

any information, apparatus, product, or process disclosed, or represents that its use would not infringe privately 

owned rights. Reference herein to any specific commercial product, process, or service by trade name, 

trademark, manufacturer, or otherwise does not necessarily constitute or imply its endorsement, recommendation, 

or favoring by the United States government or any agency thereof. The views and opinions of authors 

expressed herein do not necessarily state or reflect those of the United States government or any agency thereof.

This report is available at no cost from the National Renewable Energy

Laboratory (NREL) at www.nrel.gov/publications.

Available electronically at SciTech Connect http:/www.osti.gov/scitech

Available for a processing fee to U.S. Department of Energy

and its contractors, in paper, from:

U.S. Department of Energy

Office of Scientific and Technical Information

P.O. Box 62

Oak Ridge, TN 37831-0062

OSTI http://www.osti.gov

Phone: 865.576.8401

Fax: 865.576.5728

Email: reports@osti.gov

Available for sale to the public, in paper, from:

U.S. Department of Commerce

National Technical Information Service

5301 Shawnee Road

Alexandria, VA 22312

NTIS http://www.ntis.gov

Phone: 800.553.6847 or 703.605.6000

Fax: 703.605.6900

Email: orders@ntis.gov

Cover Photos by Dennis Schroeder: (left to right) NREL 26173, NREL 18302, NREL 19758, NREL 29642, NREL 19795.

NREL prints on paper that contains recycled content.
1

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

Cooperative Research and Development Final Report

In accordance with Requirements set forth in Article XI, A(3) of the CRADA document, this 

document is the final CRADA report, including a list of Subject Inventions, to be forwarded to 

the Office of Science and Technical Information as part of the commitment to the public to 

demonstrate results of federally funded research.

Parties to the Agreement: National Grid USA Service Company, Inc.

CRADA Number: CRD-13-535

CRADA Title: Portfolio-Scale Optimization of Customer Energy Efficiency Incentive and 

Marketing

Joint Work Statement Funding Table Showing DOE Commitment:

Estimated Costs NREL Shared Resources

Year 1 $150,000.00

TOTALS $150,000.00

Abstract of CRADA Work:

North East utility National Grid (NGrid) is developing a portfolio-scale application of OpenStudio 

designed to optimize incentive and marketing expenditures for their energy efficiency (EE) 

programs. NGrid wishes to leverage a combination of geographic information systems (GIS), 

public records, customer data, and content from the Building Component Library (BCL) to form a 

JavaScript Object Notation (JSON) input file that is consumed by an OpenStudio-based expert 

system for automated model generation. A baseline model for each customer building will be 

automatically tuned using electricity and gas consumption data, and a set of energy conservation 

measures (ECMs) associated with each NGrid incentive program will be applied to the model. The 

simulated energy performance and return on investment (ROI) will be compared with customer 

hurdle rates and available incentives to: A) optimize the incentive required to overcome the 

customer hurdle rate; and B) determine if marketing activity associated with the specific ECM is 

warranted for that particular customer. Repeated across their portfolio, this process will enable 

NGrid to substantially optimize their marketing and incentive expenditures, targeting those 

customers that will likely adopt and benefit from specific EE programs.

Summary of Research Results:

Staff in the NREL commercial buildings group and NGrid’s analytics team successfully 

collaborated to build an OpenStudio-based system called COFFEE (Customer Optimization For 

Furthering Energy Efficiency). COFFEE leveraged OpenStudio’s ability to articulate detailed 

EnergyPlus building energy models from limited input data, perform optimization-based 

calibration against monthly utility bills, and evaluate the performance of multiple energy 

conservation measures (ECMs) using commodity cloud computing resources such as Amazon’s 

Elastic Compute Cloud (EC2) as shown in Figure 1.
2

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

Figure 1: COFFEE Concept

Figure 2 illustrates the functional decomposition of the overall software system. Databases 

shown in gray are related to NGrid-specific enterprise systems, whereas the Building Component 

Library (BCL) shown in green is a publicly available source of “OpenStudio Measures” content. 

OpenStudio Measures are small self-contained scripts that can be chained together to construct 

or modify energy models. In the context of COFFEE, a combination of public and NGridspecific/private Measures were used to:

• Construct the initial model, referred to as “Model 0” (red)

• Modify Model 0 via optimization to calibrate against utility data (orange)

• Apply a wide range of NGrid-specific ECMs to the calibrated model (light blue)

Financial analysis for specific customer assessment (dark blue) was performed as a postprocessing activity. This was developed separately by NGrid and was not integrated into the 

automated system during the CRADA period of performance.

Figure 2: COFFEE Software Functional Decomposition

Input data used for the process included:

• NGrid customer data

• Consumption data

• Previously installed ECMs identified by NGrid’s “In-Demand” tracking system
3

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

• Public tax records

• Actual and Typical Meteorological (AMY/TMY) data

• ECM market pricing information

The Model 0 “expert system” combined customer-specific information including North 

American Industry Classification System (NAICS) code for building type, vintage, square 

footage, and number of floors with reference data sets that included construction, load, 

occupancy, and schedule definitions from the Department of Energy reference building models

These data sets were expanded to support building types not considered by the 16 reference 

building archetypes (e.g. funeral homes, movie theaters, etc.). HVAC systems were inferred

from a set of ASHRAE 90.1 Appendix G systems based on building type and vintage. Finally the 

model was updated with any ECMs that had previously been identified via NGrid’s In-Demand 

system. Figures 3-5 illustrate diversity in Model 0 results produced by the expert system for three 

NGrid customers. Details related to building geometry, fenestration, space-type allocations, 

HVAC systems, and energy end-uses breakdowns are specific to each building.

Figure 3: Model 0 Results for a Small Retail Customer

Figure 4: Model 0 Results for a Large Office Customer
4

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

Figure 5: Model 0 Results for a Mid-Rise Apartment Customer

Automated calibration of Model 0 against monthly consumption data utilized an optimization 

algorithm that combined genetic and gradient methods1

. ASHRAE guideline 14 for energy 

modeling was initially used to define the objective function and tolerances for optimization. 

Guideline 14 calls for +-5% net mean bias error and less than 15% coefficient of variation in root 

mean squared error against consumption data, although NGrid elected to relax these tolerances to 

reduce compute time while still producing acceptable solutions. Free model parameters used in 

the calibration process included:

• Space type ratios

• Adjust infiltration rate

• Hours of operation:

Adjust Length

Shift

• Adjust load multipliers:

Occupancy 

Ventilation

Lighting

• Adjust equipment multipliers:

Electric equipment

Cooling coil and chiller COPs

Fan efficiency and pressure rise

Gas equipment and burner efficiency

Water use

Figure 6 illustrates typical calibration results for electricity and gas consumption of the mid-rise 

apartment example above.

 1 https://cran.r-project.org/web/packages/rgenoud/index.html
5

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

Figure 6: Comparison of Modeled Predictions with Electric and Gas Consumption Data for Mid-Rise 

Apartment Example

Figure 7 contains tables for this same building that summarizes model performance relative to 

available consumption data.

Figure 7: Modeled and Measured Comparison Tables Produced by COFFEE
6

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

NGrid identified 7 ECM “families” for analysis. These encompassed 87% of NGrid’s incentive 

products, and included:

• Single package and split system unitary air conditioners

• Dual enthalpy economizer controls

• Electronically commutated fan motors

• Demand controlled ventilation

• Programmable thermostats

• Lighting system retrofits

• Lighting controls

While some of these ECMs were either applied or not (e.g. demand controlled ventilation), 

others had multiple options (e.g. single package and split system unitary air conditioners, which 

had four possible configurations). NGrid elected to generate a full design of experiments to 

capture every possible combination of ECMs, resulting in just over 1,100 derivative models per 

customer. Figure 8 shows these combinations via one of COFFEE’s built in visualizations, an 

interactive parallel coordinate plot.

Figure 8: Parallel Coordinate Plot Depicting All Possible Retrofit Alternatives for a Customer Building

While COFFEE does provide some rudimentary tools to navigate the modeled portfolio and 

associated analysis results, its real purpose is to populate an analysis database that can be 

accessed by sales tools or other enterprise systems2

. To that end, ECM analysis results are stored 

in a COFFEE NoSQL database along with the OpenStudio Measure “recipe” for reconstructing 

Model 0 and the calibrated model. Archiving the various models and design alternatives as 

references to combinations of model construction, calibration, and efficiency measures rather 

than discrete energy models allows for the degree of data compression that is required for 

COFFEE to ultimately scale to NGrid’s entire service territory.

COFFEE was deployed on Amazon EC-2 in late spring of 2015, operating on a population of 

buildings in NGrid’s Massachusetts service territory. Figure 9 shows a screenshot of the 

 2 The final step in the COFFEE process, financial assessment of ECM combinations to produce ranked, 

recommended bundles for each customer was implemented as a post-processing system by NGrid staff.

Since NREL staff were not involved in this part of the workflow, it is not within the scope of this report.
7

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

COFFEE web interface near the completion of this initial demonstration. The interactive 

dashboard summarizes progress broken down by analysis phase and building types. Clicking on 

different categories allows the user to inspect all buildings that sit in a particular analysis queue 

(e.g. Model 0 results that are awaiting calibration). Some rudimentary searching, sorting, 

reporting, and visualization capabilities were implemented in the web interface to enable 

development and debugging of COFFEE.

Figure 9: COFFEE Main Web Interface

Nearly 3,600 buildings were processed during this initial deployment that used on average 24 

EC-2 nodes with 32 cores per node. Model 0 processing required 2-3 minutes, calibration took 

25-40 minutes, and ECM analysis lasted between 70-115 minutes per building. This equated to 

around $10-20 of CPU time for each building analyzed.

Buildings in the overall population were either not processed or “errored out” due to a number of 

factors. These included:

• Invalid addresses or inactive customer account

• Incomplete customer data record

• Invalid NAICS code (e.g. NA, NotABuilding, Amusement Park, etc.)

• Missing or incomplete consumption data (e.g. customer turnover, or customer outside of 

NGrid gas franchise)

Opportunities to improve COFFEE include:

• Methods to proxy missing or incomplete data

• More robust error reporting and handling

• Queuing system improvements to further reduce human interaction with the system

• Integration of financial analysis with the cloud service for greater scalability and database 

coherence

• Development and deployment of a user interface to communicate customer opportunities 

and gather additional intelligence that can refine Model 0
8

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

As of this writing, NGrid staff are extending the population to Rhode Island and making 

additional improvements to COFFEE. NREL has granted an exclusive license for the COFFEE

source code to National Grid. COFFEE-specific Measure content and the COFFEE server code 

are not available publicly.

Subject Inventions Listing:

None

Report Date:

12/24/2015

Responsible Technical Contact at Alliance/NREL:

Dr. Larry J. Brackney

Name and email address of POC at company:

National Grid USA Service Company, Inc.

Dale Kruchten - Director: Analytics, Modeling, and Forecasting

100 E Old Country Rd.

Hicksville, NY 11801

Phone: 516-545-2434

Email: Dale.Kruchten@nationalgrid.com

This document contains NO confidential, protectable, or proprietary information.