NBS

Publications I

NAT'L INST. OF STAND & TECH

AlllOb 225475

. NBSIR 81-2346

Energy Management and Control

Systems (EMCS) User

Satisfaction Study

U.S. DEPARTMENT OF COMMERCE

National Bureau of Standards

Center for Building Technology

Building Equipment Division

Washington, DC 20234

August 1981

Department of Energy

_yr 1000 Independence Avenue

Washington, DC 20585

1UU

,u56

81-2346

1981

NATIONAL BUREAU

or STANDARDS

LIBRARY SEP i 6 1981

nOL G-CC ~ k

NBSIR 81-2346

ENERGY MANAGEMENT AND CONTROL

SYSTEMS (EMCS) USER

SATISFACTION STUDY

Gary W. Dickinson

U.S. DEPARTMENT OF COMMERCE

National Bureau of Standards

Center for Building Technology

Building Equipment Division

Washington, DC 20234

August 1981

Department of Energy

1000 Independence Avenue

Washington, DC 20585

U.S. DEPARTMENT OF COMMERCE, Malcolm Baldrige, Secretary

NATIONAL BUREAU OF STANDARDS, Ernest Ambler, Director

Abstract

A study of 86 energy management and control systems (EMCS) was made to

determine users’ satisfaction and perceived system reliability. Many WC

systems reportedly were not meeting users’ expectations for energy, dollar, or manpower savings. Nearly one-third of the EMCS users in this study were not

satisfied with their system’s performance. Undependability of the manufacturer

and proprietary hardware and software were the most common problems reported.

Factors found to have no apparent effect on user satisfaction were: (l) the

type of facility (with the exception of Hospitals) in which the EMCS is

installed, (2) which manufacturer supplied the system, and (3) whether or not

the system was owned by the Federal Government.

The data suggest that EMCS maintenance training, in-house maintenance

capability, and reduced dependence on the system manufacturer may help improve

user satisfaction.

i i i
Foreword

This is one of a series of reports documenting National Bureau of Standards

(NBS) research and analysis efforts in developing energy and cost data. These

data support the Building Energy Conservation Criteria Program sponsored by the

Office of Buildings and Community Systems, U.S. Department of Energy (DOE). The

work described in this report was supported by DOE/NBS Task Order No. A 008-BCS

under Interagency Agreement No. EA 77 A 01 6010.

iv
Table of Contents

Page

Abstract i

Foreword ii

1. INTRODUCTION 1

1 . 1 EMCS in perspective 1

1 .2 User dissatisfaction in perspective 1

2. APPROACH 2

3. PROBLEMS IDENTIFIED 2

4. CHARACTERISTICS OF SYSTEMS WITH HIGH USER SATISFACTION 4

5* SCOPE: HOW EXTENSIVE ARE THE PERCEIVED UNRELIABILITY PROBLEMS? ... 6

6. SYSTEM COMPARISONS 6

7. SUMMARY 9

8. REFERENCES 11

APPENDIX 12

v
List of Figures

Page

Figure 1 - Problem identification and extent 3

Figure 2 - Perceived unreliability vs. system characteristics 5

Figure 3 - Perceived unreliability vs. types of facilities 7

Figure 4 - Perceived unreliability vs. system age 8

vi
1. INTRODUCTION

This report summarizes the findings of a study made to determine failures

experienced with energy management and control systems (EMCS) . The goals of

this study were to determine user's satisfaction with their EMCS, and to

identify common problems and possible cures, through interviews with system

users.

Because no scientific method was used in the process of selecting the study

participants, there is no way to extrapolate this study data to represent

general users of EMC systems.

1.1 EMCS in perspective

A common goal of all EMC systems is to reduce the energy costs

associated with heating, ventilating, and air-conditioning (HVAC)*, while

meeting a pre-selected set of temperature and humidity goals. EMCS

included in this study were designed for use in applications ranging from

small buildings with as few as 50 monitor and control points, to large

multi-building complexes with as many as 15,000 points. These systems have

environmental and control sensor data brought together at a central control

area where the starting and stopping of HVAC devices can be performed.

The more advanced EMC systems also include one or more computers that

turn energy-consuming devices on or off, control dampers, decide which

source of heating or cooling is most efficient for present conditions, and

shed electrical loads to keep within peak electric demand limits.

1.2 User dissatisfaction in perspective

The information presented in this paper is not based on precise

failure data; it contains a degree of uncertainty due to user subjectivity.

For instance, two users of the same EMC system may perceive its performance

differently. Another factor that could affect integrity of the data is the

possible reluctance of users to admit failures that some people may

construe as inadequate operator skills.

No previous studies about EMCS user satisfaction have been found. However,

two guidelines [l ,2] that provide detailed approaches to planning,

specification, and installation of EMC systems are available. The U. S. Navy

guideline [l], referred to as the inter-agency guide spec., is designed to

assist all government agencies with their EMCS procurements. The Veterans

Administration guideline [2] is a research report focusing on V.A. hospital EMC

systems. Although neither guideline's primary effort is the discussion of EMCS

problems, they may, by their comprehensive approach, provide a basis for

improving performance of these systems.

Among the approaches recommended by these guidelines is the decentralizing

of system control. This "distributed processing" technique has the potential

for reducing the load on a system's main computer and may help resolve

*A glossary of MCS terms is located at the end of this report.

- 1 -
transmission link problems uncovered by this study. The "smart" field interface

devices (PID's) recommended are able to continue functioning when the main

computer fails, thus improving reliability.

Since distributed processing systems tend to be more complex and costly

than non-distributed EMC systems, only multi-building complexes or very large

building are likely to benefit from this technology. Some of these new

distributed type systems were about to come into use as this study was in

progress; but any improvement in EMCS performance as a result of this new technology will have to be determined after these systems have been operating

for some time.

2. APPROACH

A study was conducted to determine failure experiences in a large number of

EMC systems installed in diverse facilities. By contacting users close to the

system operation, problems causing user dissatisfaction were uncovered.

Discussions with EMCS users centered around the following five questions:

What are or have been the major problem areas?

Are you satisfied with your EMCS performance?

Are you using all of the functions purchased with the system?

Is EMCS maintenance done in-house or by contract?

Has your maintenance team had EMCS maintenance training?

Responses to these questions were pursued in detail. For example, a

complaint about hardware was pursued further to localize it to the computer,

peripheral devices, or sensor failure. Because of sometimes conflicting

responses, careful analysis of the data was necessaiy. "Yes, I'm satisfied with

my system's performance", may have come from the same user who responded, "We

are expanding our system with manufacturer B's equipment due to all the problems

we had with manufacturer A".

Identification of the problem areas and analysis of the study data follow.

3. PROBLEMS IDENTIFIED

Figure 1 shows the problems that were identified during the study and the

number of times each problem was reported. Since the complaints are not

completely independent, the following examples may help differentiate the

categories.

The most common problem reported was undependability of the manufacturer.

Most of the complaints concerning this problem centered around the inability of

the manufacturer to keep a system operational. Inadequate documentation, poorly

- 2

-
PROBLEM

IDENTIFIED

M FG. IS UNDEPENDABLE

PROPRIETARY PROBLEMS

SOFTWARE PROBLEMS

LIGHTNING, DATA LINK

PERIPHERALS / HDWE

GETTING SPARE PARTS

CPU

SENSORS

4 8 12 16 20 24 28

NUMBER OF COMPLAINTS

Figure 1 - Problem Identification And Extent

- 3 -
trained service people, and slow response on service calls were frequent

complaints. Other complaints included slow turn-around for software changes and

having to make repeated calls to the manufacturer to get repairs made.

Proprietary problems were reported as the most frustrating. Skilled

technicians at EMCS sites were unable to repair hardware or to make software

changes because the manufacturer refused to supply adequate documentation. In

some cases, users could not obtain hardware or software maintenance training

from the system supplier because such training would reveal proprietary

information.

Users mentioned only a few specific software problems. Among then was the

inability of the software to perform simple data gathering or averaging

routines. Other software problems were fairly general, such as intermittent

failures in system programs causing excessive downtime, or the inability to

bring optimizing programs up to an acceptable performance level.

In several cases, lightning destroyed major electronic assemblies in EMC

systems. Most other related complaints were about noise effects on the data

transmission link which resulted in significant data losses.

Peripheral device and other hardware problems were broadly distributed.

There were slightly more complaints about printers, but tape and disc drives,

loop remotes, and data gathering panels all had about the same number of reports

of failures.

Obtaining spare parts was also reported as a frustrating problem. Some

users encountered delays of several weeks in getting parts, while others

encountered delays exceeding six months before getting replacement parts for

their systems.

Most central processing unit (CPU) problems were intermittent and, like

intermittent software problems, the source was hard to locate and that caused

much system downtime. Only a few users reported specific CPU problems such as temperature sensitivity.

Complaints about sensor problems were very consistent: all were about

calibration drift.

4. CHARACTERISTICS OP SYSTMS WITH HIGH USER SATISFACTION

Data indicated that there were three major differences between systems with

high and low user satisfaction. The data in Figure 2 show user satisfaction may

be related to the three characteristics. Many more users who have had EMCS

maintenance training perceived their systems as reliable than unreliable. Also,

more than twice as many users who have in-house maintenance capability judged

their system's performance to be reliable than unreliable.

These two maintenance-related characteristics may be associated with the

third, which is the use of optimizing functions. These data show that for the

- 4

-
SYSTEM

CHARACTERISTICS

HOVE EMCS

MAINT. TRAINING

HAVE IN-HOUSE

MAINTENANCE

USE OPTIMIZING

FUNCTIONS

R = PERCEIVED RELIABLE SYSTEMS

U = PERCEIVED UNRELIABLE SYSTEMS

PERCENT OF SYSTEMS

Figure 2 - Perceived Unreliability VS. System Characteristics

- 5-
systems that have optimizing capability, more than twice as many users reported

that their systems are reliable than unreliable.

5. SCOPE: HOW EXTENSIVE ARE THE PERCEIVED UNRELIABILITY PROBLEMS? Of the 86 EMC systems included in this study, 29$ of the users were not satisfied with their system's performance. While making a comparison of the

data, several areas were investigated. These were: (l) the type of facility the system is installed in, (2) system age, (3) manufacturer, and (4) user group. Analysis of these data follows. Perceived Unreliability vs. Types of Facilities - Figure 3 - The data presented

on this graph show that all other facilities Have EMCS unreliability problems

greater than hospitals. If the hospital data were not included with the other study data, the results would show an increase from 29$ to more than 35$ of users who were not satisfied with their system's performance.

Why do the hospitals have systems which appear more reliable than those in other facilities? Studying age profile, manufacturer, and system characteristics does not account for this performance difference. One factor that contributed to hospital reliability is the V.A. "Guidelines" [2]. Increasing user awareness of latent problems and providing an effective approach for handling EMCS technology in the hospitals resulted in improved reliability.

Since V.A. hospitals make up two-thirds of the total hospital sample, the effect of their improved reliability affects results for the entire group. Perceived Unreliability vs. System Age - Figure 4 - The overall trend of this

graph shows a general increase in perceived unreliability with increasing age. This seems reasonable since failures in mechanical parts are often due to wear

and fatigue, which can increase with age. No reason has been found to explain

the break in trend shown by the highly unreliable 4-year-old systems (1976

vintage). Also, because the data for systems that are less than two years old

is based on a very small sample, those results could be misleading.

Perceived Unreliability vs. Manufacturers - The study data indicated that all manufacturers ' systems have a" similar perceived reliability record. Perceived Unreliability vs. User Groups - To compare perceived reliability

infomation the system data were separated by user group into federal military

(25 systems), federal non-military (41 systems), and non-federal (20 systems).

Comparison of these data shows that each user group had similar perceived

reliability experiences.

6. SYSTEM COMPARISONS During the study two differently structured EMCS installations were

observed to compare user experiences and system configurations. Both systems are described below and are considered to be reliable by the system operators.

- 6-
TYPES OF NO. OF FACILITIES

HOSPITALS

POSTAL SERVICE

BULK MAIL CENTERS

AIR FORCE BASES

OTHER

(UNIVERSITIES,

OFFICES, STORES)

6 12 18 24 30 36

SYSTEMS

24

18

16

28

PERCENT OF SYSTEMS THAT ARE

PERCEIVED TO BE UNRELIABLE

Figure 3 - Perceived Unreliability VS. Types of Facilities
SYSTEM AGE

IN YEARS

NO. OF

SYSTEMS

>6

6

5

4

3

2

<2

REFERENCE YEAR

IS 1980

55 'A

27 */

28 */

58 A

19 A

7 A

10 20 30 40 50 60

1 1

1 1

18

12

16

14

4

PERCENT OF SYSTEMS THAT ARE

PERCEIVED TO BE UNRELIABLE

Figure A - Perceived Unreliability VS. System Age

- 8 -
The first system is a relatively small system configured like many in the

study that do not have optimizing functions. It has remote monitoring and data

logging of control points, and is used as a manual start/stop center for singlehuilding HVAC equipment. The automatic mode is not used because the operators

anticipate heating or cooling load changes and adjust the system manually. They

are performing functions like optimum start/stop and load shedding.

One factor contributing to this system's perceived reliability is in-house

maintenance. Many replacement parts are kept on hand to assist in repairing by

replacing suspect components. The system manufacturer is only occasionally

called for repair assistance, while the system operators play the major role in

making this system reliable and effective. It should be noted, however, that

this system does not operate near its potential for automatic control with the

existing equipment.

The second system is one of the largest in the country, and is used as a

multiple-building HVAC control system. It has many optimizing functions

including optimum start/stop, load shedding, chiller/boiler optimization, and

enthalpy control. Also, a complete back-up computer and trend log are available

to ensure reliable operation.

Some of the factors that users say contribute to this system's reliability

are: (l) personnel training, (2) English language man-machine interfaces at

control consoles, (3) electric surge protection to minimize li^itning problems,

( 4 ) as-built drawings and documentation, and (5) maintenance management software

to keep track of preventive maintenance schedules.

Five full-time people are responsible for system operation during each

three-shift work day, and are assisted by the system manufacturer in maintenance

tasks. Personnel commitment, capability of using optimizing functions, and

back-up systems are all factors in this system's high reliability.

This comparison of two dissimilar systems shows that user satisfaction or perceived reliability does not depend on system size or the use of automatic

controls.

7. SUMMARY

The data show that many problems do exist with EMC systems; about one-third

of the system users in this study reported trouble keeping their system

operational. Although the study revealed no causative factors for the problems, some associative factors have been identified. Since data suggest that about

half of all identified problems are related to excessive dependence on the

system manufacturer, reducing that dependence could be a key factor in improving

EMCS reliability.

Analysis of system characteristics revealed that EMCS maintenance training

and in-house maintenance capabilities were found much more often with those

systems that users perceived to be reliable. Also, there were more than twice

as many reports of reliable systems than unreliable systems in the group of

users whose systems have optimizing capability.

- 9 -
System age was found to affect reported reliability in a logical way; that

is, perceived unreliability increases with increasing age. To minimize old age

effects on EMCS performance some users recommend that a preventive maintenance

program be implemented.

From other system data, three factors were found to have no apparent effect

on perceived reliability: (l) the type of facility (with the exception of

Hospitals) in which the system is installed, (2) which manufacturer supplied the

system, and (3) whether or not the system was owned by the Federal Government.

- 10 -
8. References

(1) Guide Specification for Energy Monitoring and Control Systems (U.S. Navy

Ref. # TS-13941 thru 50) March 1980.

(2) Guidelines for Energy Management and Control Systems for V.A. Facilities

(Veterans Administration Project #99R082) June 1979.

- 11 -
Appendix I

Glossary of EMCS Terms

Control points - switches through which remote control functions are accomplished.

Distributed processing - a technique that uses microcomputer controllers at

remote sites to collect, process, and store sensor data

until called for by the central computer.

Electric demand limit - a predetermined rate of electricity use beyond which

large additional fees are added to a customer’s

electric bill.

Enthalpy control - an optimizing function that allows use of outside air for

cooling when total heat content of the outside air is less

than that of the inside air.

HVAC - heating, ventilating, and air-conditioning.

Intermittent hardware/software problems - those problems that occur and cannot be initiated by any known sequence

randomly

of events.

Load shedding - an optimizing function that cycles electrical loads off and on to

keep energy use below the electric demand limit.

Optimizing functions - those which improve efficiency of the EMCS by automatic,

real-time adjustment of system equipment.

Optimum start/stop - an optimizing function that saves energy by starting or stopping systems after evaluating weather conditions, thermal

inertia of the buildings, and hours of occupancy.

Trend log - a display of system parameters that is updated frequently to establish

temperature and energy use trends.

Unreliability - a broad term which in this study indicates user dissatisfaction

with EMCS performance.

- 12 -
NBS-114A (rev. 2-bc) U.S. DEPT. OF COMM. 1 . BIBLIOGRAPHIC DATA

SHEET (See instructions)

4. TITLE AND SUBTITLE

PUBLICATION OR 2 REPORT NO.

NBSIR 81-2346

. Performing Organ. Report NoJ 3. Publication Date August 1981

Energy Management and Control Systems (EMCS)

User Satisfaction Study

5. AUTHOR(S)

Gary W. Dickinson

6. PERFORMING ORGANIZATION (If joint or other than NBS. see instructions) 7. Contract/Grant No. NATIONAL BUREAU OF STANDARDS

DEPARTMENT OF COMMERCE *. Type of Report & Period Covered

WASHINGTON, D.C. 20234

9.

SPONSORING ORGANIZATION NAME AND COMPLETE ADDRESS (Street, City. State, ZIP)

Department of Energy

1000 Independence Ave.

Washington, D.C. 20585

10.

SUPPLEMENTARY NOTES

| | Document describes a computer program; SF-185, FIPS Software Summary, is attached.

11.

ABSTRACT (A 200~word or less factual summary of most significant information. If document includes a significant

bibliography or literature survey, mention it here)

A study of 86 energy management and control systems (EMCS) was made to determine

users' satisfaction and perceived system reliability. Many EMC systems reportedly

were not meeting users' expectations for energy, dollar, or manpower savings.

Nearly one-third of the EMCS users in this study were not satisfied with their

system's performance. Undependability of the manufacturer and proprietary

hardware and software were the most common problems reported. Factors found to

have no apparent effect on user satisfaction were: (1) the type of facility

(with the exception of Hospitals) in which the EMCS is installed, (2) which

manufacturer supplied the system, and (3) whether or not the system was owned by

the Federal Government.

The data suggest that EMCS maintenance training, in-house maintenance capability,

and reduced dependence on the system manufacturer may help improve user satisfaction.

12. KEY WORDS (Six to twelve entries; alphabetical order; capitalize only proper names; and separate key words by semicolon s) building energy conservation systems; EMCS; energy management; energy monitoring

and control systems; HVAC controls; relibility study

13. AVAILABILITY 14. NO. OF

PRINTED PAGES

[~y~1 Uni imited

1 1 For Official Distribution. Do Not Release to NTIS 17

1 Order From Superintendent of Documents, U.S. Government Printing Office, Washington, D.C.

20402. 15. Price

Order From National Technical Information Service (NTIS), Springfield, VA. 22161 $5.00

USCOMM-DC 6043-P80
mm