• Please turn your cell phones down or off

• Share the air

• Stay in the room

• Take side conversations outside

Specifically...

• We are here today to talk about the Framework for 

Characterization, not about the specifics of how any equipment 

may be characterized.

• This is not a regulatory activity. It is voluntary.

• We will provide summary notes, but are not recording this session 

nor providing a meeting transcript.

• Per the Federal Register Notice, “Grid connected water heaters 

are subject to an ongoing rulemaking proceeding and will not be 

discussed at the meeting.”

Meeting Ground Rules
Physical Characterization of Connected 

Buildings End-Use Equipment: 

Scope and Intent

July 11, 2014 

2

Joe Hagerman

Senior Advisor, DOE-BTO
3

Physical Characterization Framework

• Expected Outcome: Work with and convene industry to develop and 

publish a national characterization framework, so that characterization 

protocols can be uniformly developed to measure the response that 

connected equipment can provide.

• Public commitment: Any activity will be developed through a series of 

public meetings.

– Open and transparent. 

• Where are we in the process? 

– First public meeting at NREL on April 30th 2014

– Second public meeting at DOE on July 11th 2014 (today!)

– Planning on release of draft document (in ~2-3 weeks)

Scope and Intent
Illustrative Public Comments

• “AHAM supports the Department of Energy (DOE) in their efforts to 

determine the value and benefits to consumers from the 

deployment of grid connected equipment and appliances... 

Objectives and goals for the physical characterization of grid 

connected appliances should include open standards and 

architectures that are expandable in the future.”

• “We (Goodman Global, Inc) strongly believe that the following 

attributes are imperative for any upcoming...initiatives: 

– It is not necessary to specify the technology within a smart system...

– ...Consumers should ultimately have the ability to override a DR signal if 

necessary. 

– The effect...on equipment reliability and system efficiency must be 

considered.”

4
Illustrative Public Comments

• “We (Massachusetts Department of Public Utilities) fully support 

DOE’s leadership on this topic...To foster widespread deployment... 

we recommend that the DOE framework be understandable and 

easily communicate the capabilities of smart appliances to 

consumers... we support the use of open source standards to the 

greatest extent possible and a framework that allows for utilities 

and third-party providers to test out many business models...”

• ”...M2M Systems Integrators strongly endorses the Green Button 

initiative, and its potential to inexpensively combine HVAC 

operations data with energy consumption and demand history for 

creating profiles to reveal relationships between facilities 

management and power usage.”

5
Public Comments - Conclusion

• General support for DOE as the convener of the industry 

stakeholders to develop characterization protocols in an open 

and transparent process. 

• General agreement that DOE is the right entity to develop 

these types of resources for the nation.

6
7

Physical Characterization Framework

• Goal for Today: DOE will present the structure and content for a draft 

framework, discuss this material at the meeting, and publish a draft 

framework document for formal comment after this meeting (~2-3 weeks)

• The Framework (and this meeting) will cover:

• Scope, Terminology, and Connected Equipment

• Characterization Protocol Framework

• Process for Developing Characterization Protocols

• To illustrate the Framework, we will also cover a notional 

example with hypothetical data for a connected room air 

conditioner.

Scope and Intent
8

Primary Objectives of the Framework

1) A characterization protocol structure and performance metrics 

that industry can use to evaluate the services that connected 

equipment can deliver. Applies to equipment in buildings: 

• Residential

• Commercial

• Industrial 

2) A process for collaborating with industry stakeholders to develop 

characterization protocols and performance metrics for connected 

equipment.

How connected equipment communicates (physical, protocol, security, 

privacy) is not within the scope of this framework

Scope and Intent
9

Why Develop a Framework?

• Promote innovation

• Help establish a scalable market for connected equipment. Data to inform: 

– Consumers/building owners

– Manufacturers

– Utilities (electric, gas)

• Protect consumer value (within limits)

– Quality of service, amenity provided by equipment

– Cost of life-cycle operation, energy minus incentives minus maintenance plus...

• Protect manufacturers

– Damage to equipment, violation of warranty

– Consumer dissatisfaction

• Inform utilities and service providers

– End-user, societal, grid, energy market services

– Create an opportunity for new services and value streams

Scope and Intent
10

DOE Working Core Principles

• Work with and convene industry around this topic and need.

– Any activity will be developed through a series of public meetings.

– Open and transparent

• Minimize the burden of characterization.

– Leverage existing test methods

• Separate characterization of physical response from that of 

communication protocols

• Drive characterization via use-cases and services

Scope and Intent
Terminology and Definitions

11
• DOE envisions a future where Connected Equipment provides 

valuable Services to consumers, building owners, third party 

providers, or electricity providers. Measuring equipment 

Responses through Characterization is an important step toward 

realizing this future.

• These terms need to be defined in order clarify this vision and to 

provide common ground for discussion.

Terminology and Definitions

12 Terminology and Definitions

Revised in response to comments from April 30th, 2014 meeting:

Smart, Grid-Connected, Actions
• “Equipment” are appliances and equipment that 

consume electricity within residential, commercial, or 

industrial buildings. 

• Examples of equipment include:

- Refrigerator - Heat pump - Vehicle charger

- Computer - Room AC - Inverter

- HVAC - Chiller - Energy storage

Equipment

13 Terminology and Definitions
• “Connected” is the capability of equipment to receive and 

transmit signals, make decisions and respond accordingly. 

Connected equipment also includes intelligent features that 

broaden the usability, impact, or capabilities of the 

equipment.

• Example of connected equipment functionality:

– HVAC equipment detects a compressor failure and transmits condition 

to a third party maintenance provider. 

– Refrigerator receives load curtailment request then adjusts electrical 

load and forecasts future availability 

Connected

14 Terminology and Definitions
Services

• “Services” are consumable and perishable functions that 

benefit interested parties.

• Services of interest for characterizing equipment may include 

those described in “Reference Guide for Transaction-Based 

Building Controls Framework ,” PNNL-23302, April, 2014.

– End-user, Energy market, Societal, or Grid 

• Example services include:

– Maintenance, diagnostics, data center queuing, energy trading

– Peak shifting, renewable integration

15 Terminology and Definitions
• “Responses” include physical or informational actions taken by 

equipment in reaction to external signals or internally 

generated information.

• Example physical responses include:

– Modify electrical load, adjust set points

– Prevent short cycling, schedule events

• Example informational responses include:

– Forecast future conditions, log performance data

– Report maintenance issues, alert consumers 

Response

16 Terminology and Definitions
Characterization

• “Characterization” involves measuring the physical and 

informational performance of equipment along with 

supporting experimental parameters.

• Characterization is not energy efficiency testing (e.g. appliance 

standards testing). 

• Examples of characterization:

- Electrical Load Voltage, current, phase angle

- Temporal Response duration, recovery time, lock out time 

- Experimental Temperature, pressure, humidity, flow rate

- Informational Operating modes, status

17 Terminology and Definitions

Physical
Public Comments

DOE envisions a future where Connected Equipment provides 

valuable Services to consumers, building owners, third party 

providers, or electricity providers. Measuring equipment 

Responses through Characterization is an important step toward 

realizing this future.

18 Terminology and Definitions

1. How can these terms be better defined? 

2. What other terms are important to define?
Characterization Protocol 

Framework

19
What is Characterization?

20 Characterization Protocol Framework

• Characterization is not...

– Annual energy efficiency testing

– Seasonally adjusted rating 

– Label development program

– Test procedure development

– ENERGY STAR

• Characterization is... 

– Evaluation of the physical and 

informational responses that 

are possible for connected 

equipment

Characterization is the first step toward:

• Mapping equipment characteristics to services

• Helping industry determine value and benefits

• Evaluating national impact of connected equipment 
Linking Characterization to Services

21 Characterization Protocol Framework

Performance 

Metrics

Characterization 

Data

Services

Characterization 

Sequence

Benefits to nation, consumers, building owners, 3rd

parties, or electricity providers

Metrics provide information for evaluating the 

ability of connected equipment to deliver services

Physical or informational data gathered during 

characterization is used to compute metrics 

Order and content of each step required for 

gathering characterization data 

Experimental Setup

Characterization

Execution
Characterization Protocol Elements:

Experimental Set-Up

22

Instrumentation

Equipment

Installation

Boundary 

Conditions

Global

Definitions

Characterization Protocol Framework

3. Are there other aspects of the experimental set-up to be 

considered for connected equipment?

Blue Elements of existing test methods

Red May differ from existing test methods

Experimental

Set-Up

• DOE anticipates that 

experimental set-up 

elements may not vary 

substantially for 

connected equipment 

• Leverage existing test 

methods and industry 

familiarity 
Characterization Protocol Elements:

Execution

23

Characterization

Data

Characterization 

Sequence

Performance 

Metrics

Characterization Protocol Framework

Responses to be 

Characterized

Determination of 

Eligibility

Blue Elements of existing test methods

Green Specific to connected equipment

Red May differ from existing test methods

Approved 

Response List

Characterization 

Execution • DOE anticipates that 

execution elements may 

vary considerably for 

connected equipment

• New elements may need 

to be developed to 

characterize connected 

equipment

4. Are there other aspects of the characterization execution to 

be considered for connected equipment?
Determination of Eligibility

• Propose that equipment have minimum features in order to 

qualify for characterization as connected equipment (go/no-go)

• For example, minimum features may include:

– Two-way digital electronic communication

– At least one automated response from an approved list of responses

5. Should there be a step to determine eligibility for 

characterization? 

6. If so, what are the minimum features?

24 Characterization Protocol Framework
Responses to be Characterized

• Example responses that could be characterized from an 

approved list of responses

Load - Adjust load, scheduling/delay

Consumer - Response thresholds and limits

Equipment - Short cycling, overloading

Reporting - Alert, history, mode, status, availability 

Applications - M&V, forecasting, diagnostics

Physical

Informational

25 Characterization Protocol Framework

Responses to be 

Characterized

7. What responses should be characterized for connected 

equipment?
• DOE proposes an approved response list 

to assure stakeholders that equipment 

responses have value and reflect services

• Developing and maintaining the approved 

list may reside within the process 

(described later) for developing 

characterization protocols

Approved Response List

8. Should there be an approved list of 

responses to be characterized?

Approved 

Response List

26 Characterization Protocol Framework

Approved 

Response List

Informational

Forecast

Status/Availability

Record history

Report/Alert

Diagnostic

Maintenance

M&V

Physical

Adjust load

Move set points

Consumer protection

Schedule events
Characterization Sequence

• The characterization sequence should be informed by services

• Sequence may depend heavily on equipment type

• Example aspects of a characterization sequence may include:

– Controlling profiles (e.g., temperature ramp, electrical load) 

– Timing of data collection (e.g. sampling intervals)

– Switching of operating modes (e.g. eco/med/fast)

– Activation of informational responses (e.g. low refrigerant, stored data) 

27 Characterization Protocol Framework

9. Does characterization sequence depend on equipment type? 

10. What aspects should be included in a characterization 

sequence for connected equipment?
Characterization Data

• Illustrative physical and informational data to be collected

11. What data should be collected for physical, informational, 

or other responses?

28 Characterization Protocol Framework

Physical Informational Other

• Voltage

• Current

• Phase angle

• Time

• Response duration

• Response time

• Recovery time

• Experimental parameters

• # of cycles

• Presence/absence

• Modes

• States

• Sensor values

• Set points

• Lock out

• Failures

• DOE seeking input
Performance Metrics

• Illustrative physical and informational metrics to be computed

29 Characterization Protocol Framework

Physical Informational Other

• kW, kWh, kVAR

• Energy consumption as 

% of baseline

• Power reduction or 

increase as % of 

baseline

• Length of response

• Availability

• Deficit of service

• Recovery of service

• Present/absent

• Correct/incorrect

• Time lag

• DOE seeking input

12. What metrics should be computed for physical, 

informational, or other responses?
Characterization Framework

30 Characterization Protocol Framework

Performance 

Metrics

Characterization 

Data

Services

Characterization 

Sequence

Experimental Setup

Characterization

Execution

Eligibility and 

Approved List
Public Comments

31

3. Are there other aspects of the experimental set-up to be considered for 

connected equipment?

4. Are there other aspects of the characterization execution to be considered 

for connected equipment?

5. Should there be a step to determine eligibility for characterization? 

6. If so, what are the minimum features?

7. What responses should be characterized for connected equipment?

8. Should there be an approved list of responses to be characterized?

9. Does characterization sequence depend on equipment type? 

10. What aspects should be included in a characterization sequence for 

connected equipment?

11. What data should be collected for physical, informational, or other 

responses?

12. What metrics should be computed for physical, informational, or other 

responses?

Characterization Protocol Framework
Illustrative Example: 

Room Air Conditioner (RAC)

32
Scenario

• ACME Equipment, Inc. approaches 

Wiley Electric Co. with a 

proposition to supply connected 

room air conditioners (RACs) to its 

customers to supply grid services.

• Wiley Electric requires 

characterization of equipment’s 

ability to respond before certifying 

their eligibility for participation in 

their programs.

• ACME Equipment submits its RAC 

to a 3rd-party testing agency to 

undergo characterization.

Illustrative Example Only

33
Confirmation of eligibility

 2-way communication (product offers WIFI/SEP 2.0)

 Automated response (product offers load curtailment)

Responses to be characterized

Load – adjusts thermostat upward 5oF when mode activated by 

<specified> SEP 2.0 command

– returns thermostat to original setting when deactivated 

by <specified> SEP 2.0 command

Consumer – load mgmt. must be enabled by user button

– 5-hr time limit on load management mode 

Equipment – mode persists for minimum of 2 minutes to prevent 

short cycling

Applications – forecasts remaining off time when in mode

– diagnostic warning for low refrigerant charge

Eligibility & Characterization

Illustrative Example Only

34
0.00

0.20

0.40

0.60

0.80

1.00

1.760

1.780

1.800

1.820

1.840

59 60 61

Real Power (W)

Power Factor

Frequency (Hz)

Consumption vs. Frequency

Input

Power Factor

0.00

0.20

0.40

0.60

0.80

1.00

1.700

1.750

1.800

1.850

1.900

110 115 120

Real Power (W)

Power Factor

Voltage (V)

Consumption vs. Voltage

Input (kW)

Power Factor

Purpose of test: Determine fundamental electrical properties 

of device and its steady-state energy performance

RAC Electric Characterization

35

Base Case

115 V

60 Hz

110 V

60 Hz

120 V

60 Hz

115 V

59 Hz

115 V

61 Hz

Voltage Test Frequency Test

59.0 59.5 60.0 60.5 61.0

1.97

1.98

1.99

2.00

2.01

2.02

2.03

110 115 120

Frequency (Hz)

Output/Input

Voltage (V)

Output/Input Ratio vs. Voltage

vs. Voltage

vs. Frequency

Illustrative Example Only

Not actual test data
RAC Load Curtailment Characterization Tests

Purpose of test: Characterize curtailment capabilities: e.g., amount of power 

curtailed, time lag, effect on energy consumption, etc.

Example Metrics

• Absolute load reduction

• % load reduction

• Time lag from request to 

curtailment

Example Metrics

• 92% recovery energy

• 5oF temp. rise limit

• 5-hr mode time limit 

(not shown) temp. limit 

reached

recovery energy = 

92% of shifted energy

baseline 

period, 

with 50% 

duty cycle

time lag = 2 min.

100% load 

reduction 

(1.8 kW) 

curtail request

Illustrative Example Only

36

Not actual test data
RAC Load Following Test

Purpose of test: Characterize load following capabilities, i.e., ability to follow 

load-up/load-down signals at increasingly short intervals (for ancillary 

services, renewables integration, etc.)

baseline period, with 50% duty cycle transition to middle of 

curtailment range

8-min. test 4-min. test 2-min. 

Failure @ 

1-min. test

Example Metrics

• 104% recovery energy (not shown)

• 100% load following, down 2-minute intervals

• Time lag (2-min)

Illustrative Example Only

37

Not actual test data
Connected Equipment is Already on the Way!

From a June 24th 2014 article in the press:

“ConEd will help NYC residents connect their air 

conditioner to their iPhone for free”

• New York utility ConEd is offering customers a free outlet adapter 

that turns a standard window-mounted air conditioner into a WiFiconnected unit that can be controlled from an iPhone.

• As part of the program, customers can sign up to receive up to five 

free kits — normally retailing for $120 each that plug in between the 

wall outlet and window unit, connecting the unit to the cloud and 

enabling remote control using a companion app.

38
Public Comment

39

13. Would it be useful to have illustrative examples like this in 

the Framework document? 

14. After seeing this illustrative example, does the Framework 

need additional steps or further revision?
Process for Developing 

Characterization Protocols

40
41

Process for Developing 

Characterization Protocols

• DOE is committed to an open and transparent process

• DOE will develop a draft proposed protocol for a particular 

type of connected equipment and circulate it for public review 

and comments

• Address and respond to the comments and release the final 

protocol in the Federal Register

Process for Developing Framework
42

Process for Developing 

Characterization Protocols (cont.)

• Alternatively, DOE can achieve the same objective through a 

consensus-like process where DOE and stakeholder 

representatives may meet as a committee to compile in a 

shorter timeframe the text of a draft characterization protocol 

for an equipment type.

• The committee may also be responsible for:

– Creating and maintaining an approved list of equipment responses

– Identifying services to which the characterization is applied

Process for Developing Framework
Public Comments

DOE is committed to an open and transparent process to engage 

industry in developing characterization protocols for connected 

equipment.

43

15. DOE requests comments on which of the two options for 

establishing the characterization protocols best 

addresses industry needs and minimizes their burdens.

16. Are there other options that DOE can pursue for 

establishing characterization protocols?

Process for Developing Framework
Overview of Questions for 

Public Comment

44
Questions

Terminology and Definitions

1. How can these terms be better defined? 

2. What other terms are important to define?

Characterization Protocol Framework

3. Are there other aspects of the experimental set-up to be 

considered for connected equipment?

4. Are there other aspects of the characterization execution to 

be considered for connected equipment?

5. Should there be a step to determine eligibility for 

characterization? 

6. If so, what are the minimum features?

45
Questions (cont.)

7. What responses should be characterized for connected 

equipment?

8. Should there be an approved list of responses to be 

characterized?

9. Does characterization sequence depend on equipment 

type? 

10. What aspects should be included in a characterization 

sequence for connected equipment?

11. What data should be collected for physical, informational, 

or other responses?

12. What metrics should be computed for physical, 

informational, or other responses?

46
Questions (cont.)

Illustrative Example - RAC

13. Would it be useful to have illustrative examples like this in 

the Framework document? 

14. After seeing this illustrative example, does the Framework 

need additional steps or further revision?

Process for Developing Characterization Protocols

15. DOE requests comments on which of the two options for 

establishing the characterization protocols best addresses 

industry needs and minimizes their burdens.

16. Are there other options that DOE can pursue for 

establishing characterization protocols?

47
Questions - Other

Other

17. What tools, approaches, and reports would help inform an 

analysis of the aggregate impacts of implementation of 

connected equipment at scale?

48

To Comment, please direct your browser to Regulations.gov 

at http://www.regulations.gov/#!documentDetail;D=EERE2014-BT-NOA-0016-0008 and hit the blue “Comment Now” 

button on the upper right-hand side. This comment period is 

open until Friday August 8 2014. DOE plans to request an 

extension of this comment period to maximize the 

opportunity for public input.
Next Steps

49
Schedule

Activity Date

1st Public Meeting: physical characterization of connected 

equipment (Golden, CO)

Apr 30, 2014

Public Comment Period on Request for Information re physical

characterization of connected equipment

Apr 8- Jun 23

2nd Public Meeting: development of a draft framework for physical

characterization of connected equipment (Washington, DC)

Jul 11

Release of draft framework document for public comment Approx. end of 

July

Public Comment Period concludes on draft framework document Approx. early

Sept.

3rd Public Meeting: Finalizing framework document and DOE 

announcement of next steps (Washington, DC)

Oct. – Nov.

Release of final framework document Nov. – Dec.