# Presentation: 351541

## Slide 1

## Agenda

- DOE Data Center Energy Efficiency initiative
- EPA Report to Congress –
- 		Server and data center efficiency
- Demonstration Projects
- Discussion/Q&A

## LBNL resources involved with DOE and EPA projects

- DOE
- Bill Tschudi
- Dale Sartor
- EPA
- Rich Brown
- Eric Masanet
- Bruce Nordman
- Jon Koomey
- Arman Shehabe
- Bill Tschudi
- Dale Sartor

- 5 Subcontractors

## DOE – Data Center Initiative

- Energy Efficiency and Renewable Energy (EERE)
- Industrial Technologies Program (ITP)
- Building Technologies Program
- Hydrogen, Fuel Cells, & Infrastructure Technologies
- Federal Energy Management Program (FEMP)
- DOE National Laboratories

## Save Energy Now Program

- Pilot Assessments of six centers – LBNL working with 5 subcontractors
- Starting with LBNL self-benchmarking protocol - refine and develop assessment tools
- Develop training for assessment contractors
- Train and qualify assessors

## Save Energy Now Program

- Solicitation - Many assessments performed by subcontractors – robust benchmark info
- Expand and disseminate best practice information
- National demonstrations of new or underutilized technologies
- Research new solutions

## Save Energy Now Program

- Coordinate with EPA Energy Star
- Collaborate with Industry
- Outreach
- Lead by example:
- Assess Federal centers
- Energy efficiency in  Federal procurement specs
- High performance computing
- Ultimately a voluntary certification program (e.g. ISO) leading to continuous improvement

## How do you participate?

- Register on web site to get regular updates
- Participate in peer review of products, protocols and best practices
- Sign up for technical working groups on web site
- Conduct self senchmarking and report results
- Use tools from LBNL site and download protocol at :http://hightech.lbl.gov/datacenters.html
- Apply for data center assessments (solicitation coming in Fall)

## Websites

**DOE**
**www.eere.energy.gov/datacenters/**
**LBNL**
**http://hightech.lbl.gov**
**http://hightech.lbl.gov/datacenters**

## Design guidelines were developed in collaboration with PG&E 

- Guides available through PG&E’s Energy Design Resources Website

## Design guidance is summarized in a web based training resource 

- http://hightech.lbl.gov/dctraining/TOP.html

## EPA Report to Congress

- HR5646 -  Public Law 109-431 requires EPA to report to congress by June 22, 2007
- Subject:  What is energy implications of Servers and data centers?  What can be done to improve efficiency?
- LBNL and EPA prepared report with input from industry through a workshop and comments to the review draft – over 50 commenters

## Recommendations - guiding principles

- Change will accelerate with top down intervention
- All areas of data center need to be addressed
- Objective, credible information is needed
- Prescriptive approaches won’t work due to rapidly changing industry
- Federal government can lead by example
- Any action should be part of a more comprehensive program for buildings and industry

## Advanced look at recommendations  (selected recommendations - subject to change pending EPA final review)

- Develop objective, credible performance rating system for data centers
- Partner with industry to develop Energy Star rating system
- Develop standards for energy management and continuous improvement
- Utilities use ratings for incentive programs

## Advanced look at recommendations  (selected recommendations - subject to change pending EPA final review)

- Challenge executive-level managers to commit to continuous improvement
- Public recognition for best performers
- Direct Federal Government CIO’s and data center operators to adopt improvement goals

## Advanced look at recommendations  (selected recommendations - subject to change pending EPA final review)

- Develop and promote use of TCO
- Require use of TCO methods in procurement and utility incentive programs
- Encourage the use of cross discipline data center management teams

## Advanced look at recommendations  (selected recommendations - subject to change pending EPA final review)

- Develop objective, and credible energy performance metrics for all data center equipment
- Work with stakeholders to develop energy efficiency metrics for servers, network equip., CRAC units, UPS, etc.
- Label better performing equipment  e.g. Energy Star ratings
- Require use of Energy Star equipment in Federal Data Centers

## Advanced look at recommendations  (selected recommendations - subject to change pending EPA final review)

- Promote new technologies and practices
- Objective and credible information on reliability and performance
- DOE sponsored demonstrations
- Develop National real-world demonstration facilities
- Publish case studies

## Advanced look at recommendations  (selected recommendations - subject to change pending EPA final review)

- Training on efficiency issues and strategies to address them
- Develop energy awareness campaign
- Identify actions that data center managers can take
- Develop training with cross functional emphasis

## Advanced look at recommendations  (selected recommendations - subject to change pending EPA final review)

- Promote use of distributed generation and combined heat and power
- Provide financial incentives
- Demonstration and education projects

## Advanced look at recommendations  (selected recommendations - subject to change pending EPA final review)

- Provide for research and development
- Performance metrics
- Improve virtualization software
- Improve software development tools to allow software to better use chip level multiprocessing
- Software efficiency
- Improve power conversion
- Improve power managment

## Advanced look at recommendations  (selected recommendations - subject to change pending EPA final review)

- Provide for research and development – continued:
- Encourage adoption of liquid cooling
- Develop active power management strategies
- Market assessment and evaluation
- Basic research (e.g. semiconductor materials)

## EPA Report - observations

- Very comprehensive considering time for preparation
- US load estimates – from Koomey report
- 1.2% of nation’s electricity
- ~ = color television electricity
- large and growing market
- Policy recommendations favor solutions developed with industry
- Voluntary programs

## EPA Website

- http://www.energystar.gov/index.cfm?c=prod_development.server_efficiency

## PG&E Sponsored Data Center demonstration projects

- Outside air economizer demonstration
- Contamination concerns
- Humidity control concerns
- “Air management” demonstration

## Encouraging outside air economizers

- Issue:
- Many are reluctant to use economizers
- Outdoor pollutants and humidity control considered equipment risk
- Goal:
- Encourage use of outside air economizers where climate is appropriate
- Strategy:
- Address concerns:  contamination/humidity control
- Quantify energy savings benefits

## Project objectives

- Identify potential failure mechanisms
- Measure contamination levels in data centers
- Observe humidity control
- Evaluate economizer effect on contamination levels
- Compare particle concentrations to guidelines
- Document economizers use in data centers

## Data center contamination guidelines

**Data center contamination guidelines**

- Limited literature connecting pollutants to equipment failure
- ASHRAE Technical Committee
- “Design Considerations for Data/Com Equipment Centers”
- Guidelines for particles, gases, humidity
- Industry Sources: Telcordia GR-63-CORE/IEC 60721-3-3
- Designed for telephone switching centers
- Based on research over 20 years old
- Primary concern: current leakage caused by particle bridging

**Notes:**
Currently I’m interested in testing that belief on data centers:
These are buildings house rows and rows of servers

Common in California
Growing
Because of all the heat for the servers, lots of energy to cool equipment

Use lots of energy, so have a pretty big environmental impact...

So what’s the air quality connections here?

## Particle bridging

- Only documented pollutant problem
- Over time, deposited particles bridge isolated conductors
- Increased relative humidity causes particles to absorb moisture
- Particles dissociate, become electrically conductive
- Causes current leakage
- Can damage equipment

**Notes:**
The main contaminant concern is the introduction of particulate matter that could deposit between isolated conductors

If hygroscopic ionic particles deposit, absorb water, and dissociate between conductors, it can produce an electrically conductive bridging between the conductors that has been shown to damage electronic equipment.  

## Particle measurements

- Measurements taken at eight data centers
- Approximately week long measurements
- Before and after capability at three centers
- Continuous monitoring equipment in place at one center (data collection over several months)

## Some reference concentrations

**Some reference concentrations**

**IBM Standard**

**EPA Annual Health Standard**

**EPA 24-Hour Health Standard**

***and***** ASHRAE Standard**

## Outdoor measurements

**Outdoor measurements**

**IBM Standard**

**EPA Annual Health Standard**

**EPA 24-Hour Health Standard**

***and***** ASHRAE Standard**

- Note scale

## Indoor measurements

**Indoor measurements**

**IBM Standard**

**EPA Annual Health Standard**

**EPA 24-Hour Health Standard**

***and***** ASHRAE Standard**

- Note scale

## Indoor measurements

**Indoor measurements**

- Note scale

## Data center w/economizer

**Data center w/economizer**

- Note scale

**EPA Annual Health Standard**

**EPA 24-Hour Health Standard**

***and***** ASHRAE Standard**

## Improved Filtration

- Filter Efficiency

## Humidity measurements  without economizer

**Humidity measurements  ****without economizer**

**ASHRAE Recommended Upper Limit**

**ASHRAE Recommended Lower Limit**

**ASHRAE Allowable Lower Limit**

**ASHRAE Allowable Upper Limit**

## Humidity measurements  with economizer

**Humidity measurements  ****with economizer**

**ASHRAE Recommended Upper Limit**

**ASHRAE Recommended Lower Limit**

**ASHRAE Allowable Lower Limit**

**ASHRAE Allowable Upper Limit**

## Findings

- Water soluble salts in combination with high humidity can cause failures
- It is assumed that very low humidity can allow potentially damaging static electricity
- ASHRAE particle limits are drastically lower than one manufacturer’s standard
- Particle concentration in closed centers is typically  an order of magnitude lower than ASHRAE limits
- Economizers, without other mitigation,  can allow particle concentration to approach ASHRAE limits
- Filters used today are typically 40% (MERV 8) efficiency

## Next steps: building the case for  air economizers  

- Analyze material captured on filters
- Collaborate with ASHRAE data center technical committee
- Contamination book
- Electrostatic discharge
- Collect failure data and study failure mechanisms
- Evaluate improved filtration options
- Develop recommendations

## “Air Management” demonstration

- Goal:
- Demonstrate better cooling and energy savings through improvements in air distribution in a high density environment.

## Demonstration description

- The as-found conditions were monitored
- Temperatures
- Fan energy
- IT equipment energy
- Two high-intensity rows and three computer room air conditioning units were physically isolated from rest of the center – approximately 175W/sf

## Demonstration description, con’t

- Two configurations were demonstrated
- Air temperatures monitored at key points
- IT equipment and computer room air conditioner fans energy were measured
- Chilled water temperature was monitored
- Chilled water flow was not able to be measured

## First configuration - cold aisle isolation

## Second configuration – hot aisle isolation

## Demonstration procedure

- Once test area was isolated, air conditioner fan speed was reduced using existing VFD’s
- Temperatures at the servers were monitored – top, middle, bottom of rack
- IT equipment and CRAC fan energy were monitored
- Chilled water temperatures were monitored
- Hot aisle return air temperatures were monitored – ΔT was determined

## Fan energy savings – 75%

**Since there was no mixing of cold supply air with hot return air - fan speed could be reduced resulting in large fan energy savings**

## Temperature variation improved

## Better temperature control would allow raising the temperature in the entire data center!

- ASHRAE Recommended Range

- Ranges during demonstration

## 

Discussion/Questions??

**Discussion/Questions??**

- William Tschudi
- wftschudi@lbl.gov