ENERGY STAR Servers
Version 2.0
Draft 2 Webinar
May 24, 2012
RJ M
Meyers, US E
Environmental
ilP
Protectioni A
AgencyEvan Haines, ICF International
Al Thomason, TBWC, LLC

Bryan Berringer, U.S. Department of Energy

AgendagTime ((all EST))

Topicp

11:00 AM

Introduction

11:10 AM

Updates from SERT Development Effort

11:45 AM

PPDS: Revised Format
Break ­ Lunch

12:10 PM
12:40 PM
1:15 PM

Testing and Blade Servers
Idle Requirements

1:45 PMDefinitions, Other Topics

1:55 ­ 2:00 PM

Timeline and Closing2Version 2.0 Goals• Revise Product Family Structure• Evaluate Blade Servers• Continue to push Standard Information
Reporting (PPDS)• Generate public Active Mode efficiency()
dataset for future use (SERT)
3

Server Efficiency Rating Tool (SERT)Development
KlKlaus-DieterDi t L
Lange
SPECpower Committee Chairman
Standard Performance Evaluation Corporation

4

SERTOverview• Consistent measure of p
performance and ppower for all
candidate ENERGY STAR servers on various load levels• Significant improvements over prior art (SPECpower_ssj2008)– GUI to assist with setup and execution– Broader coverage of system loads and stress of components viavarious synthetic worklets:• Processor• Memory• In-frame storage– Design for power / performance modifier– Enhanced automated validation of results

5

SERTDevelopment Milestones
SERT Milestone

StatusAlpha Release

Successfully Complete

Review Period and Development

Successfully Complete

Beta 1 Release

Successfully Complete

Review Period and Development

Successfully Complete

Beta 2 Release

Sign up starts 12th June 2012

Review Period and Development

TBD

Release Candidate (RC) Release

TBD

FinalTBD

6

SERT
Status Update
Near Complete• Measurement framework• Worklet Candidates for:– CPU– Memory– Storage– Hybrid CPU/Memory• GUI• Extensive testing• Documentation

In Progress• Results representation• Final Worklet selection• Recommendation for:– allowable or required
tuning parameters– number of Java Virtual
Machines (JVM)• Fi
Finall UI• HW / SW discovery for allenvironments to assist
with setup and report

7

SERT
Worklet Candidates
Workload

WorkletAlpha

Beta 1

Beta 2

RC1

CPU

CPU C
CPU_Compress

I l d dIncludedI l d dIncluded

TBD

TBD

CPU

CPU_CryptoAES

Included

Included

TBD

TBD

CPU

CPU_SOR

Included

Included

TBD

TBD

CPUCPU_SHA256--

TBD

TBD

CPU

CPU_FFT

Included

Included

TBD

TBD

CPU

CPU_LU

Included

Included

TBD

TBD

CPU

CPU_XMLvalidate

Included

Included

TBD

TBD

MemoryMem Flood
Mem_Flood

Included

Included

TBD

TBD

MemoryMem_XMLvalidate1

Included

Included

TBD

TBD

MemoryMem_XMLvalidate2

Included

Included

TBD

TBD

Storage

Storage_Random-Included

TBD

TBD

Storage

Storage_Sequential-

Included

TBD

TBD

Storage

Storage_Mixed

Included

Included

TBD

TBD

HybridHybrid_CSSJ

Included

Included

TBD

TBD

Idle

Idle

Included

Included

Included

Included

Please see the SERT Design Document for detail description of each worklet: http://www.spec.org/sert/docs/SERT-Design_Doc.pdf

8SERTDirect Current (DC) Support•

SERT is neither supported nor tested with DC loads today andcurrently no resources are devoted to including this support.

•The SPECpower Committee is in favor of including DC support andthe decision to start the implementation of DC measurements could
be positively influenced by companies whose focus is DC computing
by making volunteers available to the SPECpower committee.

•Besides code changes
changes, significant effort would have to be spent
defining acceptance criteria for DC power analyzers and theevaluation of uncertainty calculation for DC measurements, as wellas testing and documenting themthem.

•Later this year the subcommittee will create a detailed plan foradding DC support in order to accurately state the additional
req ired resources.
requiredreso rces
9

SERT
Metric / Score / Platform• Each worklet will p
produce a measure representing
pg theperformance achieved by the SUT as well as the average
power consumption at multiple target load levels. An overall
provided and not recommended.
score is not p• Complexity of performance and power measures acrosscomponents at multiple target load levels makes creation of asinglei l metrict i diffi
difficult.
lt• The available resources enabled SERT to be implementedand tested on the following platform / OS (64-bit(64 bit only)
combinations:Platform

X86 (AMD)X86 (AMD)X86 (Intel)

X86 (Intel)

Itanium

PowerOSWindows Server
2008 R2

LINUXWindows Server
2008 R2

LINUX

HP-UX 11i

AIX10SERTGraphical User Interface (GUI)• GatheringGg SU
SUT hardware and software configurationg• Archiving the measured results and log files• Setup and Executing• Default Mode
– EPA compliant test record– Executes the entire SERT suite• Advancedd a ced Researchesea c Modeode– Subset of workloads and worklets– Customization of worklets
11

SERT
Beta 2•

Head start on expertise needed to qualify servers for ENERGYSTAR V2 specification•Gain experience to better comment on the next iteration of Version 2draft•

Help SPEC to identify and resolve problems prior to your ENERGYSTAR qualification•

Open to all ENERGY STAR Stakeholders– Must have accepted power analyzer and temperature sensor(http://www.spec.org/power/docs/SPECpower-Device_List.html)– Must commit to providing feedback on SERT to SPEC and ENERGYSTAR to support further development– Must commit to share numerical results with EPA and SPEC (will not bemade ppublic)
blic)
12

SERT
Q&A

sertsupport@spec org
sertsupport@spec.org
www spec org/sert
www.spec.org/sert
SPEC, the SPEC logo and the tool and benchmark names SERT and SPECpower_ssj2008 are registered trademarks of SPEC.
Copyright © 1995-2012 Standard Performance Evaluation Corporation (SPEC) All rights reserved. Reprint with permission.

13

SERT and Servers v2.0
• EPA will require SERT testing
testing, reporting ofinformation• No active mode levels set in v2.0
– Will be major objective of v3.0

• Results reported on ENERGY STAR
website,, also in PPDS
14

For Review• SERT Design Document:– http://www.spec.org/sert/docs/SERTDesign Doc pdf
Design_Doc.pdf

15

PPDS: Introducing a New Format
RJ Meyers
US Environmental Protection AgencygyMeyers.Robert@epa.gov

16

Reportingpg Requirementsq• Under Version 1.0,, manufacturers are required
qtogenerate a Power and Performance Datasheet toaccompany ENERGY STAR qualified Servers

Development Activities Interactive Comparison Tool
• EPA is working
g to develop
p a more centralized anduser-friendly format for the requirement• As part of the Uninterruptible Power Supplies
effort, EPA has shared updates on development ofan approach to allow electronic display of PPDS
data and eliminate the use of “loose”
loose Excel-based
Excel basedfiles• Interactive “widget” that will allow the publication ofqualified products’ performance information in an
y accessible,, dynamicyformat
easily18

PPDS Data Submission
Qualified Product Exchange (QPX)

Power and Performance Data Sheet Widget

EPA-recognizedEPA
i d certification
tifi tibodies submit data for PPDS

AAggregatedt d data
d t forf onlineli didisplay
l

CB 1

CB 2
QPX

CB 3 …

Benefits• Data centralized and associated with other
ENERGY STAR product data– Update once,
once appears everywhere• Widget design allows forf incorporation
both on the ENERGY STAR web site andon Partner sites

20

Screenshots – UPS Tool

21

Screenshots – UPS Tool

22

Screenshots – UPS Tool

23

Screenshots – UPS Tool

24

Testing and Blade Servers
Bryan Berringer
US Department
pof EnergygyBryan.Berringer@ee.doe.gov

25

Overview• This section will cover both ggeneral Test
Method items and Blade Server
considerations• Many of the key points of discussion for
Blades overlap with Test Method revisions for
Draft 2
• Revised Chassis and Idle requirements for
Blades in next section
26

Introduction – Program History• Draft 1 specification released April 9, 2010
– Blade servers incorporated into the specification– Active mode power – for data collection• Additional test method drafts released for data collection– March 11, 2011 and August 9, 2011
– Changes based on stakeholder comments– Divide by N instead of N-1 method to calculate single blade power• Draft 2 specification released May 11, 2012
– Draft 2 Test Method - revision based on August 9, 2011 draft

Introduction – Testing Summary• 4 servers tested for idle mode power consumption
ProductForm Factor

Processor

Operating SystemSSingleSSocket
ServerSTower

X86-6486 6• WindowsTwo Socket ServerRackX86-64• Windows• LinuxTwo Socket ServerRack

X86-64• Windows• LinuxBlade ServerBladeDraft 2 Test Method – Anticipating SERTTesting• Language
g g added to harmonize with Server Efficiency
yRating Tool (SERT)– Power meter• CCalibrated
lib t d byb a standard
t d d nationaltil metrology
t liinstitutetit t (e.g.,(NIST)• Logging – At least 1 set of data measurements per second• Set is defined as watts– Temperature sensor• Overall accuracy of ± 0.5 °C or better• Minimum reading
g rate of 4 samples
pp
per second

Blade Testingg• The Draft 2 Test Method reflects
procedures distributed with the V2 dataset
development in 2011
• Clarifications
Cl ifi ti
incorporated
it dbbasedd onfurther feedback from stakeholders

30

Recommendation 1Populating Half Blade Chassis• The Test Method includes provisions for Full- and HalfChassis Testing• Draft 1 Test Method – HalfHalf-Chassis
Chassis testing case
– Fill top row of the chassis first and then proceed
downwards– Fill partiallyti ll populatedl t d rows ffrom centert outwardstd• Issue– May end up operating power supplies in low efficiency
conditions due to partial population of power domain

Recommendation 1
Populating Half Blade Chassis• Proposed change– Follow
F llmanufacturer
f trecommended
d d approachh– If user manual recommendation is not available:• Completely populate one power domain before proceeding to thenext• Fill partially populated power domains from center outwards• Advantages– Real world configuration– Improves power supply efficiency– Fewer power supplies used

Recommendation 1Populating Half Blade Chassis - Example
Blade Configuration – 8 blades
Slot 1

Slot 2

Slot 3

Slot 4

Slot 5

Slot 6

Slot 7

Slot 8

Powero e Domain
o a 1Powero e Domain
o a 2

Draft 1 Test Method
Slot 1

Slot 2

Slot 3

Slot 4

Slot 5

Slot 6

Slot 7

Slot 8Draft 2 Test Method
Slot 1

Slot 2

Slot 3

Slot 4

Slot 5

Slot 6

Slot 7

Slot 8

Recommendation 2Idle Power Test
• Draft
D ft 1 T
Testt Method
M th d– Measure idle power after the completion of workload run• Issue– UUT may not enter idle state of operation due tob kbackgroundd processes anddddaemons iinitiatediti t d bby tthe
hworkloads

Recommendation 2Idle Power Test
• ProposedPd changeh– Measure idle power before engaging workload• Advantages– UUT will be in idle state throughout the power
measurement period

Follow Up
Divide by N Method for Per Blade Power• N-1 method p
provided inconsistent results– Per blade power consumption varied with blade position
Table 1: Single Blade Power consumption (N-1) method
Scenario #Single Blade AC Power (kW)10.18620 176
0.176

30.196• ~ 12% variation in idle power measurements

Follow Up
Divide by N Method for Per Blade Power•Divide by N method– Power consumed by the whole system divided by thenumber of blades populated– Observed 4% variation in per blade power
T bl 2:Table
2 Si
Singlel Blade
Bl d Power
PConsumption
Cti
dividedi id byb N method
th dChassis configuration•Single Blade AC Power (kW)

Full Chassis

0.181

Half chassis

0.189

AdAdvantages
t– Amortize chassis overhead across installed blades– Reduces testing burden for stand alone chassis powerIdle Power Requirements
Evan Haines
ICF Internationalevan.haines@icfi.com

38RequirementqSummaryy
1S/2S

3S/4SBladeBldServersIdle – Levels

Yes

Idle
Disclosure

Yes

Yes

Yes

Full Power –Disclosure

Yes

Yes

Yes

SERT

Yes

Yes

Yes

39

1S/2S• Adders changes incorporated:– Memory: Per-GB memory adder reduced after evaluation ofcomponent-level data provided by stakeholders and the Draft 2dataset• SSystemst
presentt ini theth dataset
d tt thatth t hadh d Idle
Idl PPower more ththan offset
ff t bbyeligible Version 1 adders– Redundant PSU: TBD• EPA is seeking feedback on alternatives to the flat 20 W adder present
in Version 1 (i.e., an appropriate value that scales with functionality)40

1S/2S• Base Idle levels are maintained from
Version 1 in recognition of the currentpp
y for these levels todataset and opportunity
continue to recognize more energypefficient products41

3S/4S and Blades• Some server manufacturers pprovided data for
products in these categories with great effort• Unfortunately,
Unfortunately the dataset does not support settingof an Idle Power level applicable to the market asa whole• With the implementation of the revised productfamilyy approach,pp, EPA believes that Version 2 will
generate a fuller picture of power demands acrossranges of similar platforms and allow for better
gof levels
investigation
42

Blades• Chassis
Chassis-level
level Power Requirements– Requirements for power limits on chassis
overhead removed• Included as placeholders in previous documents• “Divide by N” testing approach amortizes chassis
overhead into blade assessment• Overhead functionality hosted uniquely to different
blade system designs (chassis vs. in-blade)– Thermal Management and identification criteria
remain, along with general PSU criteria
43GPGPUs• Stakeholders raised the topic of GPGPUs (and later,expandabled bl compute capability
bili viai adddd iin cards)d )• Section 3.9.1 included a requirement to test with andwithout GPGPUs installed– Will allow the program to evaluate the Idle Power impact ofsuch features• Further feedback encouraged from stakeholders:– Similar technologies that may require investigation• (The approach proposed for GPGPUs is not intended for broadapplication)– Assessment and Test procedures beyondyIdle
measurements.
44

Definitions and Other Topics
Evan Haines
ICF Internationalevan.haines@icfi.com

45

Definition Changeg Summaryy• Computer Servers– An exemption from the core requirement for ECC memory inServers is proposed for systems offering greater than 50 nodes• Resilient Server– EPA understands that stakeholders are actively developing amore cohesive set of criteria describing Resilient Servers. EPA
plans to evaluate the resulting proposal once available andwelcomeslcommentst on thisthi iissue• High Performance Computing (HPC) System– EPA intends to remove the definition unless further developmentof requirements requires a description of HPC systems

46

Other Topicsp• Solidifying on Five-Point
Five Point Product Family test
approach– Balance between testing burden and creation ofproduct dataset• Adding Time Stamping to reported
Environmental Data (Power andTemperature)– Seeking Stakeholder input on this potential
addition47

Closing48

Timeline

TopicTimeframe

Draft 2 Distributed

May 1111, 2012

End of Draft 2 Comment period

June 6, 2012

SERT Beta 2 launch

Mid-June 2012

Draft 3 (TBD)

Early July

End of Draft 3 Comment period (TBD)

Early August

Final Draft

Late Augustg

Final Specification

Early September49

References and Resources• ENERGY STAR Servers specification
revision:
www energystar gov/RevisedSpecs (clickwww.energystar.gov/RevisedSpecs
on Computer Servers)• SERT Design Document:– http://www.spec.org/sert/docs/SERT­
Design_Doc.pdf
50

Thank you!
Bryan Berringer, DOE
Bryan.Berringer@ee.doe.gov

Robert Meyers, EPA
Meyers.Robert@epamail.epa.gov]

Akshay Odugoudar, NavigantAkshay.Odugoudar@navigant.com

Evan Haines, ICF International
ehaines@icfi.com
Bizhan Zhumagali, ICF International
bzhumagali@icfi.com

servers@energystar gov
servers@energystar.gov