NREL is a national laboratory of the U.S. Department of Energy, Office of Energy Efficiency and Renewable Energy, operated by the Alliance for Sustainable Energy, LLC. 1

Reimagining Building Sensing 

and Control

Luigi Gentile Polese, Commercial Buildings Research

National Renewable Energy Laboratory

Keynote, Sensor Expo & Conference, 

Rosemont, Illinois, June 2014 NREL/PR-5500-62354
2

Sensors and The Built Environment

Abigail Credits: Dennis Schroeder, NREL 29603; Marjorie Schott, NREL; 

Carken, NREL 11348;
3

Where the US Energy Goes

2011 Total Primary Energy 

Consumption: 98 Quads

28%

32%

Industry

Transportation

40%

?

Industry: 31 Qbtu

Transportation: 27 Qbtu

Credits: Jeff Smith, NREL
4

Where the US Energy Goes

2011 Total Primary Energy 

Consumption: 98 Quads

Industry: 31 Qbtu

Transportation: 27 Qbtu

Buildings: 40 QBtu

40%

28%

32%

Industry

Transportation

Buildings

Credits: Jeff Smith, NREL
5

US Energy Consumption

40%

Buildings

22%

Residential

18%

Commercial

Credits: Jeff Smith, NREL
6

US Energy Consumption

40%

Buildings

22%

Residential

18%

Commercial

Credits: Jeff Smith, NREL
7

Market Opportunity for Building Sensors

Battlefield: 

4.6 M commercial buildings 

smaller than 50,000 square ft. 

consume 9.2% of the US 

primary energy

4.9 M

commercial 

buildings in US

81.1 B square ft.

commercial 

floorspace

Source: DOE EIA CBECS, 2003, http://buildingsdatabook.eren.doe.gov
8

Market Opportunity

Sensors and controls for Building 

Energy Management Systems

• Will top $4 billion in 2020

• 18% CAGR worldwide between now and 2020

• Market growth fueled by advanced technologies, 

price pressure, policy

Source: Lux Research, http://www.luxresearchinc.com/news-and-events/press-releases/read/sensors-and-controls-building-energy-management-systems-top-4
9

Favorable Conditions for Sensors

• The US DOE on a path to boost uptake of 

building energy efficiency through codes and 

standards

• The EU on a path with similar aggressive 

efficiency goals (20% better efficiency by 2020)
10

Leveraging the Market Opportunity

• Advanced, smart sensors

• Non-invasive

• Plug-and-play

• Cost effective

Credit: iStock 
11

Advancing Building Sensor Technology

Credit: iStock; Credit for bottom right photo: Al Puente, NREL 01320
12

Advancing Building Sensor Technology

Credit: iStock; Credit for bottom right photo: Al Puente, NREL 01320

• Smarter

• Ubiquitous

• Networked
13

Advancing Sensor Data Analysis

Credit left: Dennis Schroeder, NREL 21736; Credit right: Warren Gretz, NREL 12764

Digital readout of a chiller Building EMS — where 

sensors meet controls
14

Advancing Sensor Data Analysis

Credit left: Dennis Schroeder, NREL 21736; Credit right: Warren Gretz, NREL 12764

Digital readout of a chiller Building EMS — where 

sensors meet controls

• Sensor virtualization

• Enable new control 

applications
15

Buildings Sensor Research

Credit: Dennis Schroeder, NREL 24417; Credit small photo: Sandia National Laboratories, NREL 03637
16

Virtual Sensors

• Provide data that is hard or impossible to 

measure with physical sensors

• Detect and compensate for sensor data errors, 

drifts, or faults occurring in harsh operating 

environments

Image credit: iStock
17

Virtual Sensors

• Low-cost alternative to specialized sensing 

equipment (when it exists)

• Based on existing physical sensor hardware

• Can provide real-time outputs even if physical 

sensors provide delayed responses

Image credit: iStock
18

Model

Physical sensor

Physical sensor

Physical sensor

Physical sensor

. . . . . 

Dynamic Model

Physical quantities Virtual quantities

. . . . . To the Control 

. . . . . . . . . . System
19

Model

• Neural networks

• Multivariate 

statistical analysis

• Data fusion

• Image analysis

Physical sensor

Physical sensor

Physical sensor

Physical sensor

. . . . . 

Dynamic Model

Physical quantities Virtual quantities

. . . . . . . . . . . . . . . 

To the Control 

System
20

Ventilation & Indoor Air Quality

Credit: iStock
21

Ventilation & Indoor Air Quality

• Conventional CO2

sensors expensive, 

slow, error prone

• Solution?

Credit: iStock
22

Ventilation & Indoor Air Quality

• Conventional CO2

sensors expensive, 

slow, error prone

• Solution? A CO2

virtual sensor

Credit: iStock
23

CO2 Virtual Sensor

Sensors: the 

building Wi-Fi access 

points and routers 

infrastructure

Illustration: Marjorie Schott, NREL 
24

CO

2 Virtual Sensor

Sensors: the 

building Wi-Fi access 

points and routers 

infrastructure

Illustration: Marjorie Schott, NREL; picture at left: Luigi Gentile Polese, NREL 
25

CO

2 Virtual Sensor

Sensors: the 

building Wi-Fi access 

points and routers 

infrastructure

Sense what?

mobile devices

Credit: NREL 24505

Illustration: Marjorie Schott, NREL 
26

Image Sensors

Remote sensing, sensor fusion, virtual sensing

Image credit: iStock
27

Market Opportunity

Automotive:

– Lane Assist

– Self Parking

– Drowsiness Alert

– Customized Driver 

Settings (seat, rear & 

side mirrors, ...)

Image credit: iStock
28

Gesture Recognition Sensors

Image credit: iStock
29

Gesture Recognition Sensors

Credit: Roy Shilkrot, MIT Media Lab

http://www.morethantechnical.com/2013/05/26/extending-the-hand-tracker-with-snakes-and-optimizations-w-code-opencv/
30

Augmented Reality Sensors

Physical or virtual sensors coupled with real 

time imaging

Image credit: Larry Brackney, NREL
31

Lighting System M&V

Image credit: Larry Brackney, NREL
32

Ventilation Flow Diagnostics

Image credit: Larry Brackney, NREL
33

AR Sensor Architecture for Buildings Apps

Building 

system 

sensors

Building 

system 

actuators

Building Energy 

Management 

System (EMS)

AR App Server

BACnet

Connection

AR Client App

Wireless 

Network

Connection

Phone Image: Larry Brackney, NREL; Computer: iStock; Control Panel: Warren Gretz, NREL 12762 
34

AR Sensor Architecture for Buildings Apps

Building 

system 

sensors

Building 

system 

actuators

Building Energy 

Management 

System (EMS)

AR App Server

BACnet

Connection

AR Client App

Wireless 

Network

Connection

Phone Image: Larry Brackney, NREL; Computer: iStock; Control Panel: Warren Gretz, NREL 12762 

Integration of

– AR Technologies

– Building Systems

– Mobile Computing 

Capabilities

– Spatial Location

– Advanced Database 

Architectures

– Modern EMS 

Systems
35

Image Processing Occupancy Sensor

IPOS

o Aligns with open embedded systems and future sensor platform needs

o Example of reuse of embedded systems originally destined to other uses

o What it is

– Remote sensor/sensor fusion/virtual sensor

– Multiple outputs at once

– Embedded system

– Integrated communication with Building Automation Systems

Credit: Dennis Schroeder, NREL 24417
36

Image Processing Occupancy Sensor

Building 

system 

sensors

Building 

system 

actuators

Building Energy 

Management 

System (EMS)

Robust Algorithms 

for Finely Grained:

• Lighting Controls

• Daylighting

• Set Point 

Management

• Demand 

Controlled 

Ventilation • Open Embedded OS

• OpenCV Computer 

Vision Library

• Open BACnet Comms

• IPOS Algorithms

Top and bottom left: Dennis Schroeder, 24420, 24418 NREL; Control Panel: Warren Gretz, NREL 12762 

IPOS
37

IPOS: Fusion of Image Sensing

Motion People

Faces

12

3

Top images: Alex Swindler, NREL; Bottom image: Luigi Gentile Polese, NREL 
38

Motion Analysis
39

Illuminance and Location Awareness

Credits: Luigi Gentile Polese, NREL
40

Sensor in a Sensor

IPOS Image segmentation/multiple zones 

Simultaneous virtual sensing

Credits: Left image: NREL 14840; Right picture: NREL 15439; and Luigi Gentile Polese, NREL
41

IPOS Sensor Commissioning

Credit: Mike Linenberger, NREL 16224
42

IPOS Sensor Commissioning

Credit: Mike Linenberger, NREL 16224

Virtual Zone 1

Virtual Zone 2
43

IPOS Virtual Sensor Model

Physical Sensor

In-memory 

Buffers

Space Illuminance Virtual Sensing 

Integration

Occupant Count & 

Location

Image Acquisition

Occupant Activity

Face Detection

People Detection

Motion Detection

• Human Occupancy

• Confidence Levels

• Number & Location of Occupants

• Occupant Activity Levels

• Illuminance/Light Meter

• Light Dimming Power Levels

• Multiple Zones Outputs

Virtual Outputs
44

Higher Accuracy = Higher Energy Savings

Energy Savings %

60.3

87.8 85.9

41.2

80.3

75.6

15.5

54.1

43.3

0

10

20

30

40

50

60

70

80

90

100

Weekdays (6AM-6PM) Weekdays (6PM -9PM) Weekend (7AM-7PM)

Theoretical Best IPOS PIR (15min)
45

Sensors Needs

What will the future sensors have in common?

Open embedded system 

platforms as a need for 

advanced sensor products

• Multi-function, modular

• Open source, open license

• Embedded signal processing

• Interoperable, can be easily 

integrated into applications

• Integrates with other sensors 

and control systems

• Standard communication 

systems

Business & market 

opportunities

• Robustness, aggregation, 

multi-function, enables 

virtual sensing, localized 

intelligence

• Cost effective

• New capabilities and 

advanced applications

• Better/more energy 

efficiency enabler

Credit: Dennis Schroeder, NREL 24417
46

Challenges Ahead

Conflicting requirements: Higher Complexity, Higher 

Sensor Density, Computation-Intensive vs. Low-Power 

Consumption

Credit: Michael Sheppy, NREL
47

Challenges Ahead

A new/different workforce skillset for sensor manufacturers

Image credit: iStock
48

Challenges Ahead

Image credit: iStock
49

Opportunities from Future Sensors

Better energy efficiency: 

Renewables adoption more 

favorable/more cost 

effective

First photo: Dennis Schroeder, 21482; Second photo: SunPower, NREL 13339
50

What’s Ahead?

•Self-Organizing sensor networks

•Self-Commissioning – Optimal 

energy efficiency over building 

lifetime & Lower operational 

costs

•Self-Cooperating sensors

•Self-Decision-Making

•Self-Diagnosing, Self-Recovering, 

Self-Reporting – Lower 

maintenance costs

•Open, embedded sensor 

platforms

Image credit: iStock
51

What’s Ahead?

Image credit: iStock

Smarter, digital, networked, secure sensing
52

Thank you!

luigi.gentile.polese@nrel.gov