# Presentation: 125337

## PNNL Data Center Virtual Environment

- Guy C. Wilcox
- Systems Engineer
- Information Technology Services Division
- Pacific Northwest National Laboratory

- PNNL-SA-55732

## Key Topics

**Key Topics**

- Why Virtualize
- An overview of virtualization
- The multi-phased Virtualization project
- Virtual Infrastructure at PNNL
- The future of the Data Center
- Lessons Learned
- Questions and Audience Participation

## PNNL

**PNNL**

- A multi-program research laboratory: 4,200 staff
- $730M annual research business volume
- 8,000 networked devices
- Two large computer centers
- Hundreds of servers

**Notes:**
PNNL has a unique operating contract with DOE that enables private research.
EMSL is a unique user facility.

## The Problem Oversubscribed Data Center resources

**The Problem**** ****Oversubscribed Data Center resources**

- Improvements needed in the enterprise data center
- Most AC power circuits loaded near breaker rating
- Cooling units overworked
- Years of cabling accumulation under 1ft. false floor
- No additional UPS capacity
- Unmanaged growth in amount of servers
- No space for additional servers

**Notes:**
Explain how Data Center evolution, growth, and insufficient oversight caused this.

## The Unplanned Outage

**The Unplanned Outage**

- When: July 25, 2005
- What: An overloaded main breaker in the enterprise datacenter tripped... three times
- Why: Lack of clearly defined policy for server provisioning and data center growth
- Approximately 10% non-critical servers were shut down for 3 days
- Electrical work to replace breaker
- Redistribution of power – 3 separate UPS units

**Notes:**
Explain that we talked to system owners to decide on how to deal with the need for Electrical repairs and decided on a number of specific servers/services that we could live without until the building owners had the electrical services redistributed.

## Our Response

**Our Response**

- Data Center cabling cleanup
- Formal Provisioning policy
- Cultural challenge
- Resource upgrades
- HVAC
- UPS
- Building AC Power upgrade
- Inventory reduction
- Data Center management system (proposed)
- Server Virtualization

**Notes:**
DC mgt.  Includes the DCR (explain it)  tighter reign of the CMB.
DC cleanup cabling under floor allowing for better cooling etc.

## Virtualization Overview

**Virtualization Overview**

- VMware, Inc.

- Virtualization is an abstraction software layer that decouples physical hardware from the operating system.
- http://www.vmware.com/overview/

## Virtualization Benefits

**Virtualization Benefits**

- Reduced physical server count
- Fast provisioning of new servers
- Templates and Cloning
- Zero-downtime hardware maintenance
- Scalability and expandability
- Streamlined disaster recovery / ease of virtual machine (VM) recovery
- Positive ROI
- Physical procurement/maintenance cost reduction
- Increased uptime
- Reduction in use of data center resources

**Notes:**
Fast Provisioning.. Due to use of Templates
Zero-downtime hardware maintenance, upgrades and host OS patching

## Initial Virtualization Project Plan

**Initial Virtualization Project Plan**

- Driver: Oversubscribed datacenter resources
- Target: 10% of inventory
- Timeframe: 3 months (November 2005 – January 2006)
- Phased Approach
- Data gathering
- Infrastructure build-out
- Initial conversions (consultant lead)
- Complete conversion (PNNL staff)

## Virtualization Project First Steps

**Virtualization Project First Steps**

- Infrastructure build-out
- Architectural planning
- Conversion methodology
- Candidate evaluation
- Politics and customer service
- Identify candidate servers
- Team Building
- Training
- Conversion
- Upgrade hardware or P2V
- Coordinating outages
- Gaining customer confidence

**Notes:**
Team: Proj mgt, Network Engineers, System Engineers/Admins, SAN Engineers/Admins

## Virtual Infrastructure Planning

**Virtual Infrastructure Planning**

- Separating users from the hardware mindset
- Hardware refresh or Virtual Machine
- Which hardware to use for hosts
- What determines requirements
- Blades vs. configurable rack-mount servers
- Up-front costs and considerations
- Software licensing
- OS licenses
- VMware or Microsoft Virtual Server

## Results of the First Phase Virtualization Project (4/2006)

**Results of the First Phase ****Virtualization Project ****(4/2006)**

- Production-designated virtual environment
- The numbers:
- 4 farms (clusters), 12 hosts, 66 VMs
- 27 Physical to Virtual conversions
- 5 templates

## Phases 2 and 3

**Phases 2 and 3**

- Virtualization, phase 2 (May/June 2006)
- Assessment of approximately 400 servers as candidates for virtualization
- Stabilization of virtual infrastructure
- Results: Cooler Data Center with power-to-spare
- Virtualization, phase 3  (Current)
- Virtualization of as many servers as is feasible and affordable

**Notes:**
Need phase 2 completion statistics

## Growth of PNNL Virtual Environment

**Growth of PNNL Virtual Environment**

**Notes:**
Currently have 25 hosts Most are Dell 2850 & 2950 class servers, and over 120 vms. Most running W2k3 some RH Linux.

## Slide 15

## Realized Benefits

**Realized Benefits**

- Technology acceptance
- Customers see the possibilities and value
- Manageability – beyond our expectations
- Virtual Infrastructure Client (VMware VirtualCenter)
- Easy console access
- Fast VM deployment
- Dynamic resource allocation
- Upgrade host hardware without down time
- VMotion

## Lessons Learned 

**Lessons Learned **

- Risk of proliferation due to ease of provisioning
- Physical to Virtual issues and successes
- Various methods
- Not all servers are good candidates
- Linux administrative skill set required
- Virtual Center limitations
- Selection of appropriate host hardware
- Blades vs. configurable rack mount servers
- Network interface cards, HBAs, RAM, and CPU

## Thank You!

**Thank You!**

- Questions?
- Your Experiences...
- Guy Wilcox
- _[guy.wilcox@pnl.gov](mailto:guy.wilcox@pnl.gov)_
- (509) 375-5974