Energy Implications of Economizer Use in California Data Centers

*Arman Shehabi^1^, Srirupa Ganguly^2,\*^, Kim Traber^3^, Hillary
Price^3\
^Arpad Horvath^1^, William W Nazaroff^1^, Ashok J Gadgil^2^,*

^1^*Department of Civil and Environmental Engineering, University of
California, Berkeley, CA 94720*

*^2^Indoor Environment Department, Lawrence Berkeley National
Laboratory, Berkeley, CA 94720*

*^3^Rumsey Engineers, Oakland, CA 94607*

^\*^*Author to whom correspondence should be addressed E-mail:
<SRGanguly@lbl.gov>*

# 

# Abstract

In the US, data center operations currently account for about 61 billion
kWh/y of electricity consumption, which is more than 1.5% of total
demand. Data center energy consumption is rising rapidly, having doubled
in the last five years. A substantial portion of data-center energy use
is dedicated to removing the heat generated by the computer equipment.
Data-center cooling load might be met with substantially reduced energy
consumption with the use of air-side economizers. This energy saving
measure, however, has been shown to expose servers to an
order-of-magnitude increase in indoor particle concentrations with an
unquantified increase in the risk of equipment failure. An alternative
energy saving option is the use of water-side economizers, which do not
affect the indoor particle concentration but require additional
mechanical equipment and tend to be less beneficial in high humidity
areas. Published research has only presented qualitative benefits of
economizer use, providing industry with inadequate information on which
to base their design decisions. Energy savings depend on local climate
and the specific building-design characteristics. In this paper, based
on building energy models, we report energy savings for air-side and
water-side economizer use in data centers in several climate zones in
California. Results show that in terms of energy savings, air-side
economizers consistently outperform water-side economizers, though the
performance difference varies by location. Model results also show that
conventional humidity restrictions must by relaxed or removed to gain
the energy benefits of air-side economizers.

# 

# Introduction

Data centers are computing facilities that house the electronic
equipment used for data processing, networking and storage. Rapid growth
in computational demand emerging from various sectors of the economy is
causing strong rates of increase in servers and IT-related hardware (IDC
2007). Server performance has doubled every two years since 1999,
leading to increasingly higher densities of heat dissipation within data
centers (Belady 2007). A substantial proportion of energy consumption in
data centers is dedicated to the cooling load associated with electronic
power dissipation (Tschudi et al. 2003). A recent study estimates that
US data centers account for 61 billion kWh or 1.5% of the nation's
annual electricity consumption (US DOE 2007a). This corresponds to an
electricity bill of approximately \$4.5 billion in 2006 (EPA 2007). The
environmental impact is substantial because 70% of the electricity in US
is generated in power plants that burn fossil fuel (EIA 2007). Improved
data center cooling technologies have the potential to provide
significant energy savings. Cost savings and environmental benefits
might also accrue.

A typical data center consists of rows of tall (2 m) cabinets or racks
in which the servers, data storage and networking equipment are
vertically arrayed. The cooling of data-center equipment is accomplished
using computer room air conditioners (CRACs), which supply cold air to a
raised-floor plenum beneath the racks. The CRAC system air handler is
placed on the data center floor while chilled water in transported from
compressor-based chillers to the CRAC cooling coils. More efficient
cooling systems employ low outside air temperatures to reduce chiller
load. Cooling towers that use ambient air to directly cool or precool
the chilled water are known as water-side or fluid-side economizers.
This type of system has been claimed to cut cooling-energy costs by as
much as 70% (ASHRAE HVAC Fundamentals Handbook 2005) during economizer
operation. Based on local weather data in San Jose, water-side
economizers can be used for more than one-third of the year (PG&E 2006).
An alternate data center arrangement uses air-handling units (AHU) and
an air-side economizer. Such systems directly provide outdoor air for
cooling whenever the temperature of outside air is lower than the
set-point for return-air temperature in the data center. In San
Francisco's cool climate, outside air could contribute to some level of
air-side cooling for nearly all hours of the year (Syska Hennessy 2007).
The use of air-side economizers brings with it an associated concern
about contamination including moisture from humidity that may possibly
threaten equipment reliability. Deliquescent sulfate, nitrate and
chloride salts, in a humid environment (\> 40% relative humidity) can
cause corrosion, accumulate and become conductive, and may lead to
electrical short-circuiting (Rice et al. 1981; Sinclair et al. 1990;
Litvak et al. 2000). In this paper, the energy implications of a data
center using a CRAC system will be compared with alternative cooling
systems using air-side or water-side economizers for five different
California climate zones. The modeling results and discussion focus on
understanding the energy implications for both type of economizers and
their effectiveness in different climate zones. The equipment
reliability concerns associated with air-side economizers are
acknowledged to be important, but addressing it is beyond the scope of
the present paper.

# 

# Methods 

## Data center design scenarios

Energy-use simulations were performed for three different data center
HVAC design scenarios (Figure 1). The baseline case considers a data
center using conventional "computer room air conditioning" (CRAC) units.
In this scenario, CRAC units are placed directly on the computer room
floor. Air enters the top of a CRAC unit, passes across the cooling
coils, and is then discharged to the underfloor plenum. Perforations in
the floor tiles in front of the server racks allow the cool air to exit
from the plenum into the data-center room. Fans within the computer
servers draw the conditioned air upward and through the servers to
remove equipment-generated heat. After exiting the backside of the
server housing, the warm air rises and is transported to the intake of a
CRAC unit. Most air circulation in the baseline scenario is internal to
the data center. A small amount of air is supplied through a rooftop AHU
to positively pressurize the room and to supply outside air for
occupants. Cooling is provided by a water-cooled chiller plant.
Refrigerant in the chillers is used to cool water through heat
exchangers at the evaporator. The chilled water is then piped to the
CRAC units on the data center floor. Waste heat from the chiller
refrigerant is removed by water through heat exchangers in the
condenser. Condenser water is piped from the cooling towers, which cools
the water through interaction with the outside air. This baseline design
is common to most mid- to large-size data centers (Tschudi et al. 2003;
Rumsey 2005; Syska Hennessy 2007).

The water-side economizer (WSE) scenario assumes a CRAC unit layout
similar to that of the baseline case, except that additional heat
exchangers are installed between the condenser water in the cooling
towers and the chilled water supplied to the CRAC units. Under
appropriate weather conditions, the cooling towers can cool the
condenser water enough to cool the chilled water in the CRAC units
directly, without operating the chiller plant. The CRAC units and
chiller plant are assumed to be the same as in the baseline scenario.

The air-side economizer scenario (ASE) requires a different type of air
delivery than typically found in a data center with conventional CRAC
units. AHUs are placed outside of the data center room, commonly on the
rooftop, and air is then sent to and from the computer racks through
ducts. A ducted air delivery system creates greater air resistance than
a conventional CRAC unit layout, though this system better prevents cold
and warm air from unintentionally mixing within the data center. When
the outside air temperature is equal to or below the temperature of the
air supplied to cool the server, the AHU can directly draw outside air
into the data center and exhaust all of the return air after it has
passed across the computer servers. The movement of 100% outside air
through the system can require more fan energy than the baseline case,
as the economizer design requires more ducting, which increases air
resistance through the system. However, during this 100% outside air
mode the cooling is provided without operating the chiller, chilled
water pumps, condenser water pumps, or the cooling tower fans. Outside
air is also provided instead of recirculated air whenever the outside
air temperature is greater than the supply air temperature but lower
than that of the return air. Under this condition the chiller must
operate, but the cooling required of the chiller is less than in a case
with complete recirculation.

## Energy modeling protocol

For each design scenario, the model calculations assume a 30,000 ft^2^
(2800 m^2^) data center with an internal heat density of approximately
80 W/ft^2^ (0.86 kW/m^2^; 2.4 MW total) This size and power density are
characteristic of data centers evaluated in previous studies (Shehabi et
al. 2008; Greenberg et al. 2006; Tschudi et al. 2003). The size of data
centers varies greatly; 30,000 ft^2^ is within the largest industry size
classification, which is responsible for most servers in the US (IDC
2007). Power density in data centers is rapidly increasing (Uptime
Institute 2000) and a power density of 80 W/ft^2^ is currently
considered to be of low- to mid-range (Rumsey 2008).

Basic properties of the modeled data center for all three scenarios are
summarized in Table 1. Energy demand is calculated as the sum of the
loads generated by servers, chiller use, fan operation, transformer and
uninterruptible power supply (UPS) losses, and building lighting. The
chiller encompasses coolant compressor, chilled water pumps, condensing
water pumps, humidification pumps, and cooling-tower fans. Energy demand
for servers, UPS, and lighting are constant, unaffected by the different
design scenarios, but are included to determine total building-energy
use. The base case and WSE scenarios assume conventional humidity
restrictions recommend by ASHRAE (ASHRAE 2005). The ASE scenario assumes
no humidity restriction, which is an adjustment required to gain ASE
benefits as is typical in ASE implementation (Rumsey 2008). Air-side
economizers also require a different air distribution design and the fan
parameters associated with each design scenario are listed in Table 2.
The properties of other pumps and fans throughout the HVAC system remain
constant for all three scenarios. Values are from previous data-center
energy analyses (Rumsey 2008; Rumsey 2005).

The energy modeling approach used in this study applies a previously
used protocol (Rumsey 2008; Rumsery 2005) and is based on a combination
of fundamental HVAC sizing equations that apply equipment size and
efficiencies observed through professional experience. Building energy
modeling is typically performed using energy models such as DOE-2, which
simultaneously models heat sources and losses within the building and
through the building envelope. However, models such as DOE-2 are not
designed to incorporate some of the HVAC characteristics unique to data
centers. Also, data centers have floor-area-weighted power densities
that are 15-100 times higher than those of typical commercial buildings
(Greenberg et al. 2006). This allows accurate modeling of data-center
energy use to focus exclusively on internal heat load and the thermal
properties of outdoor air entering the building. This is the approach
taken in this study, as heat generated from data center occupants and
heat transfer through the building envelope are negligible relative to
the heat produced by servers. The building envelope may influence the
cooling load in low-density data centers housed in older buildings that
have minimal insulation. Evaluating this building type is worthy of
exploration, but the required analysis is more complex and outside the
scope of the present paper.

Both air-side and water-side economizers are designed to allow the
chiller to shut down or reduce chiller energy load under appropriate
weather conditions. Less overall energy is required for operation when
the chiller load is reduced, but chiller efficiency is compromised.
Changes in chiller efficiency used in this analysis are shown in Figure
2, representing a water-cooled centrifugal chiller with a capacity \>
300 tons and condenser water temperature of 80 °F. A chilled water
temperature of 45 °F, which is standard practice for data center
operation, is used in the base case and ASE scenario. The WSE scenario
uses a chilled water temperature of 52 °F, which is common when using
water-side economizers. This increases needed airflow rates but allows
greater use of the water-side economizers. The curves are based on the
DOE2.1E software model and apply coefficients specified in the
Nonresidential Alternative Calculation Method (ACM) Approval Manual for
the 2005 Building Energy Efficiency Standards for Residential and
Nonresidential Buildings (CEC 2005).

Annual data center energy use is evaluated for each of the three
configuration scenarios assuming that a data center building is located
in each of the five cities shown in Figure 3. Weather conditions at each
city are based on hourly DOE2.1E weather data for California climate
zones (CEC 2005).

# Results and Discussion

Results from each scenario modeled are presented in Table 3 as a
"performance ratio" which equals the ratio of total building energy
divided by the energy required to operate the computer servers. Lower
value of the performance ratio implies better energy utilization of the
HVAC system. The performance ratio for the base case is 1.55 and, as
expected, is the same for all the cities analyzed, since the operation
of this design is practically independent of outdoor weather conditions.
The base case performance ratio is better than the current stock of data
centers in the US (EPA 2007; Koomey 2007) because the base case
represents newer data centers with water-cooled chillers, which are more
efficient than the air-cooled chillers and direct expansion (DX) cooling
systems found in older data centers.

The performance ratios for the ASE and WSE scenarios show air-side
economizers consistently provide savings relative to the base case,
though the difference in savings between the two scenarios varies. It is
important the note that even small changes in the performance ratio
results in significant savings, given the large amount of energy used in
data centers. For example, reducing the performance ratio at the model
data center in San Jose from 1.55 to 1.44 represents a savings of about
1.9 million kWh/y, which corresponds to a cost savings of more than
\$130,000/y (assuming \$0.07/kWh).

Figure 4 shows the disaggregation of the cooling systems' annual energy
use, normalized by floor area, for each modeled data center by location
and design scenario. The annual energy use dedicated to the servers,
USP, and lighting is 584, 95, and 9 kWh/ft^2^, respectively. These
energy values are independent of the climate and HVAC design in scenario
and not included in the graphs in Figure 4. Economizer use is typically
controlled by combination of outside air temperature, humidity, and
enthalpy; however results shown in Figure 4 are for economizer use
controlled by outside air temperature only. Results show that the ASE
scenario provides the greatest savings in San Francisco while Fresno
provides the least ASE savings. Sacramento benefited the most from the
WSE scenario while minimal savings were realized in Los Angeles and San
Francisco. The San Francisco WSE scenario, where significant gains would
be expected because of the cool climate, is hindered by chiller
part-load inefficiencies. The relatively higher moisture content in the
San Francisco air increases the latent cooling load in the model and
causes the chiller plant to reach the capacity limit of the first
chiller more often, activating a second chiller. The second chiller
shares the cooling load equally with the first, resulting in a
transition from one chiller at a high load factor (efficient operation)
to two chillers at slightly above half the load factor (less efficient
operation). The results from the WSE scenario in San Francisco emphasize
the need for engineers to model the hour-by-hour load, rather than just
the peak load, and to size chillers such that all active chillers at any
moment will be running near their most efficient operating point.

Figure 5 shows that removing the humidity restrictions commonly applied
to data centers is necessary to gain ASE energy savings. As the relative
humidity (RH) ranged is narrowed, energy use from the fans begins to
sharply increase, surpassing the equivalent baseline energy in most of
the cities. Humidity levels are often restricted in data centers to
minimize potential server reliability issues. ASHRAE's guidelines
released in 2005 for data centers provide a "recommend" RH range between
40-55% and an "allowable" range between 20-80% (ASHRAE 2005). There is
minimal cost in applying the more conservative ASHRAE RH restrictions in
conventional data center design, such as the baseline in this study
shown in Figure 5. The influence of humidity on server performance,
however, is poorly documented and the need for humidity restrictions is
increasingly being questioned (Fontecchio 2007). The energy saving
difference between adhering to ASHRAE's recommend RH range versus the
allowable RH range is substantial, and warrants further investigation.

# Conclusion

Employing the energy-saving measures evaluated in this paper would
require a shift in conventional data center design and operation.
Various operational concerns must be addressed before widespread
adoption of these technologies could be expected in data-center
buildings. This paper contributes to the informed implementation of
air-side and water-side economizers by assessing the energy benefits of
adopting these efficiency improvements. Air-side economizers are shown
to consistently outperform water-side economizers in California, though
the difference in performance varies by the climate conditions of the
locations evaluated. Furthermore, the models show that conventional
humidity restrictions must by relaxed or removed to substantially
realize the energy benefits of air-side economizers. As the data center
economy continues to rapidly grow, energy efficiency will continue to
emerge as an important financial and environmental concern. The results
presented here contribute to our understanding of different design
implications and should assist decision makers in the implementation of
energy-efficient data centers.

# Acknowledgments

We thank Rumsey Engineers for their assistance in running energy models
and data center operators that helped determine conventional HVAC
parameters. This project was funded by Pacific Gas & Electric and by the
University of California Energy Institute, California Studies Grant
Program. Some of this work was performed at LBNL under the U.S.
Department of Energy Contract No. DE-AC02-05CH11231.

# 

# References

\[ASHRAE\] American Society of Heating, Refrigerating, and
Air-Conditioning Engineers, Inc 2005. "Design Considerations for Data
and Communications Equipment Centers." Atlanta, GA.

\[ASHRAE\] American Society of Heating, Refrigerating, and
Air-Conditioning Engineers, Inc. 2006. "Liquid Cooling guidelines for
Datcom Equipment Centers." ASHRAE Datacom Series, Atlanta, GA

\[ASHRAE\] American Society of Heating, Refrigerating, and
Air-Conditioning Engineers, Inc. 2005. " ASHRAE HVAC Fundamental
Handbook." Atlanta, GA

Belady, C. 2007. "In the Data Center, Power and Cooling Costs More than
the IT Equipment it Supports.\" Electronics Cooling Magazine. February
2007.

\[CEC\] California Energy Commission (CEC) 2005. "Nonresidential
Compliance Manual For California\'s 2005 Energy Efficiency Standards."

\[EIA\] Energy Information Administration 2007. "International Energy
Outlook, Energy Information Administration (EIA)." May 2007.

\[EPA\] Environmental Protection Agency 2007. "Report to Congress on
Server and Data Center Energy Efficiency." August 2.

Fontecchio, M. 2007. "Data center humidity levels source of debate."
SearchDataCenter.com June 18.

Greenberg, S., Mills, E., Tschudi, W., Rumsey, P., Myatt, B. 2006. "Best
practices for data centers: Results from benchmarking 22 data centers."
Proceedings of the 2006 ACEEE Summer Study on Energy Efficiency in
Buildings. Asilomar, CA.

\[IDC\] International Data Corporation 2007. "IDC's Worlwide Installed
Base Forecast, 2007- 2010." Framingham, MA

Koomey, J. 2007. "Estimating total power consumption by servers in the
U.S. and the world." Oakland, CA: Analytics Press.

Litvak, A., Gadgil, A.J., Fisk, W.J. 2000. "Hygroscopic fine mode
particle deposition on electronic circuits and resulting degradation of
circuit performance: an experimental study." Indoor Air 10: 47-56.

\[PG&E\] Pacific Gas and Electric 2006. "High Performance Data Centers."
A Design Guidelines Sourcebook. January.

Rice, D. W., Cappell, R. J., Kinsolving, W., Laskowski, J. J. 1980.
"Indoor corrosion of metal." Journal of Electrochemical Society 128:
891-901.

Rumsey Engineers. 2008 "Network Appliance: Building 02 Datacenter."
Pacific Gas and Electric Non-Residential New Construction Incentive
Program. January.

Rumsey Engineers. 2005 "Network Appliance Building 11 Datacenter."
Pacific Gas and Electric Industrial Savings by Design Program. June.

Sinclair, J.D., Psota-Kelty, L.A., Weschler, C.J., Shields, H.C. 1990.
"Deposition of Airborne Sulfate, Nitrate, and Chloride salts as It
Relates to Corrosion in Electronics." Journal of the Electrochemical
Society 137:44

Shehabi, A., Horvath, A., Tschudi, W., Gadgil, A., Nazaroff, W. 2008.
"Particulate Matter in Data Centers. " Atmospheric Environment, draft
submission, October.

Syska Hennessy Group 2007. "The Use of Outside Air Economizers In Data
Center Environments." White paper 7.

Tschudi, W.F., Xu, T.T., Sartor, D. A., Stein, J. 2003. "High
Performance Data Centers: A Research Roadmap." Lawrence Berkeley
National Laboratory. Berkeley, CA

Uptime Institute 2000. "Heat Density Trends in Data Processing, Computer
Systems, and Telecommunications Equipment." Santa Fe, NM.

\[US DOE 2007a\] United States Department of Energy 2007a. "Annual
Energy Outlook 2007, with Projections to 2030." Washington, DC

Table 1: Data Center Characteristics Common to All Design Scenarios

![](media/image1.wmf){width="3.2909722222222224in"
height="2.198611111111111in"}

Table 2: Data Center Fan Properties

![](media/image2.wmf){width="5.396527777777778in"
height="1.7277777777777779in"}

Table 3: Ratio of Total Building Energy to Computer Server Energy

![](media/image3.wmf){width="5.539583333333334in"
height="1.1756944444444444in"}

Figure 1: Schematic of Data Center Cooling Design Scenarios

![](media/image4.wmf){width="6.809027777777778in"
height="6.336111111111111in"}

Air and water flow schematic for the basecase and water-side economizer
scenarios (above).\
Air and water flow schematic for the air-side economizer scenario
(below).

Figure 2: Assumed Part Load Performance of Data Center Chillers

![](media/image5.wmf){width="4.634722222222222in"
height="2.7805555555555554in"}

Part load efficiencies for a water-cooled centrifugal chiller with a
capacity\
\>300 tons and an condenser water temperature of 26.7 °C (CEC, 2005))

Figure 3: Evaluated Climate Zone Locations

![](media/image6.wmf){width="3.6444444444444444in" height="3.375in"}

Figure 4: Disaggregated Energy Use (Climate Dependent Values Only)

![](media/image7.wmf){width="2.125in" height="1.125in"}
![](media/image8.wmf){width="2.8006944444444444in"
height="2.907638888888889in"}

![](media/image9.wmf){width="2.8006944444444444in"
height="2.813888888888889in"}
![](media/image10.wmf){width="2.8006944444444444in"
height="2.907638888888889in"}

![](media/image11.wmf){width="2.8006944444444444in" height="2.84375in"}
![](media/image12.wmf){width="2.8006944444444444in"
height="2.9694444444444446in"}

Figure 5: Chiller and Fan Energy Resulting from Humidity Restrictions

![](media/image13.wmf){width="2.625in" height="1.125in"}
![](media/image14.wmf){width="3.0506944444444444in"
height="2.872916666666667in"}

![](media/image15.wmf){width="3.0506944444444444in"
height="2.814583333333333in"}
![](media/image16.wmf){width="3.0506944444444444in"
height="2.814583333333333in"}
![](media/image17.wmf){width="3.0506944444444444in"
height="2.872916666666667in"}
![](media/image18.wmf){width="3.0506944444444444in"
height="2.751388888888889in"}