Internet Topics

Internet Topics: Matsue Data Center Park Performance and Future Initiatives
n Matsue Data Center Park Performance
Matsue Data Center Park in Matsue, Shimane Prefecture (henceforth Matsue DCP) is Japan’s first container-based data center with anoutside-air cooling system, offering high server density and easy scale-out to meet the needs of the age of cloud computing. It has beenin stable operation as the infrastructure for the GIO cloud service since operation commenced in April 2011, with containers housing
several hundred servers being transported and installed there one after another. As of July 2012 there are 13 containers in operation(Photo 1). If expansion continues at this pace, the maximum capacity of 24 containers will be installed there by the end of this fiscal year.
The “IZmo”*1 (Photo 2) IT modules are containers that do not fall under the buildings subject to the technical advice issued by the Ministryof Land, Infrastructure, Transport and Tourism in March 2011, so they do not require planning permission. This simplifies the process ofadding or removing modules. It also used to be necessary to unpack, rack-mount, and wire a large number of servers one-by-one at datacenters when constructing systems for providing cloud services. There were issues with amount of time this process required, and with theneed to secure a large amount of temporary space to store servers before mounting as well as empty boxes. Container-based data centers
drastically reduce the amount of time required to set up servers at a data center, and eliminate the need for storage and work space bymounting several hundred servers in IT modules at the server factory and transporting these by truck for installation.
Another feature of the Matsue DCP is its outside-air cooling system. Because this system is effective at reducing cooling power
consumption, which at data centers is second only to the amount consumed by IT equipment, it has been implemented at a large
number of data centers in the United States, such as those for Yahoo! and Facebook. On the other hand, it was rarely implemented inJapan because it is difficult to control, and there were design constraints associated with the need to create large openings in buildings
to take in outside air. IIJ constructed a proof-of-concept facility comprised of an IT module and a cooling module in central Japan, and ran
tests over the course of a year to establish an outside-air cooling control system, before deciding to implement outside-air cooling at theMatsue DCP. At Matsue it is possible to use outside air directly for cooling for about two thirds of the year, achieving a major reduction inpower consumption. Because this demonstrated significant energy savings on a commercial level, we believe that more and more data
centers will use outside-air cooling systems in Japan in the future.
The cooling module transitions control between three modes according to the temperature and humidity of outside air, keeping thetemperature of each server’s air inlet in the IT module within the recommended temperature and humidity requirements issued by
ASHRAE (The American Society of Heating, Refrigerating and Air-Conditioning Engineers) in 2008. In all outside-air operating mode,outside air is supplied to servers as-is in spring and autumn, and server exhaust heat is also expelled outside as-is. In mixed operating
mode, server exhaust is mixed with outside air when the outside air temperature is cold, supplying servers with air at a suitabletemperature to prevent condensation. When humidity is low, humidification is also carried out to prevent static electricity. In circulation
operating mode, because outside air cannot be used in the summer season when air is hot and humid, server exhaust air is cooled via
the direct expansion coils used in standard cooling, and then supplied to servers.
The PUE (Power Usage Effectiveness) metricproposed by The Green Grid is commonly used
as an indicator of data center power efficiency,
and this is calculated using the following formula.
PUE=1 is theoretically the lowest (best) value,
when zero power is consumed for cooling, etc.

IT equipment power consumption+power consumption for cooling, etc.

Overall data center
power consumption
PUE =IT equipment power consumption=IT equipment power consumption

Generally, PUE is higher (worse) in the summer season when a large amount of power is consumed for cooling, and lower (better) inother seasons, so it is necessary to assess it as an annual average. Because Matsue DCP is a module-based data center, and IT modules
were added in stages, over a year’s worth of operational data has been measured for two modules as of July 2012. Figure 1 shows trends
in actual values for pPUE (partial PUE, which does not take power loss for shared areas into consideration) over the course of a year for
one of these modules. In Figure 1, the pPUE for the summer season at around 1.3 is higher than other seasons. This is due to the use of
circulation operating mode, which does not use outside air and consumes more power. In comparison, because outside-air operation modeis used in spring and autumn, employing outside air as-is and reducing power consumption, the pPUE falls below 1.1. Mixed operating
mode is used in the winter season, mixing IT module exhaust air and outside air to achieve a suitable temperature, and reducing power
consumption to end up with a pPUE below 1.1. This results in an average pPUE of 1.17 for the year. pPUE does not include the loss for shared
areas such as lighting and electrical equipment, so the overall PUE for the data center will be higher (worse) than 1.17. However, we estimate

Emergency
Generators
Administration
BlockElectrical
Equipment Block
IZmo:IT Modules
Cooling
Modules
Cooling
ModuleBoundaryFenceIZmo:IT Module

Photo 1: Matsue Data Center Park

*1

Photo 2: IZmo and Cooling Module“IZmo” IT modules are container-based modules developed by IIJ for constructing data centers optimized for building cloudinfrastructure (www.iij.ad.jp/DC/technology/izmo.html) (in Japanese).

38

n Reducing PUE from 1.2 to 1.0
In the past it was conventional wisdom that data center temperatures be maintained at around 20 degrees Celsius. However, topromote energy savings ASHRAE relaxed the environmental requirements for data centers drastically in 2011 by raising the acceptabletemperature for a server’s air inlet to around 40 degrees Celsius. As a result more and more manufacturers are announcing servers
guaranteed to operate in high-temperature environments.

Internet Topics

that even with this included an annual average value of less than 1.3 is achievable, so we can assert that significant power savings havebeen made over conventional data centers, which have a PUE of around 2.

When this kind of server is used, an annual average PUE of 1.1 would be achievable. This is because it would allow cooling using
only outside air even in the summer season. As can be seen from the pPUE trends in Figure 1, if it were possible to use all outside-air
operating mode instead of circulation operating mode in the summer season, the annual average pPUE would fall below 1.1, bringing
a PUE of less than 1.2 within reach even when taking power and losses for shared areas into consideration. This would also eliminate
the need for direct expansion coils and compressor units in cooling modules, allowing for a simple fan-only configuration, and further
reducing the costs associated with cooling devices.
This summer we are planning to conduct proof-of-concept tests at the Matsue Data Center Park through the integrated operation of IT
equipment such as servers and facilities that include cooling. In these tests we will aim to achieve higher energy savings by resolving
the issue of server fan speed increasing as temperatures rise, consuming more energy. This leads to less reduction in total energy
consumption even when cooling energy consumption is lower. We hope to solve this problem through the integrated management andcontrol of IT and facilities.
We believe it will be possible to approach a PUE of 1.0 by further integrating IT and facilities. For example, if we can create sufficient
airflow to cool servers via the chimney effect using the heat generated by high-density servers, we can reduce the fans in coolingequipment and servers, eliminating the power required for cooling. There are a number of possible methods for implementing chimney
data centers using this system, such as installing containers around a large chimney, or arranging racks around chimneys inside abuilding (Figure 2).
n The Future of IT and Facility Integration
Data centers were once “facilities” for housing the “IT equipment” assets of customers, but in the cloud computing era they consist of
cloud infrastructure integrating the “IT equipment” and “facilities” of cloud providers. The integrated construction and managementof IT and facilities, including operation in high-temperature environments and chimney data centers, has enabled innovation that was
not previously possible.
In the future, instead of single data centers, multiple data centers located both home and abroad will be connected via network andoperate as one massive computer cluster. Once this becomes a reality, it will be possible to achieve redundancy through software
instead of hardware, perhaps reducing the need for individual
data center reliability to be as high as up to now. If it is possible
Chimneys
to provide a service with data centers implementing mutual
backups, it will only be necessary to maintain the battery
power required to shut down servers safely when a power
outage occurs at a data center, and generators will not beneeded. This means that data center equipment configurations
will change drastically from how they are now.
We also believe that there will be calls to integrate smart grids
into data centers that consume a large amount of power, toutilize a limited amount of power efficiently and grow the ITindustry. IIJ will continue to pursue development of technologythat goes beyond simple energy savings, such as controlling IT
loads between multiple data centers on a network according
to the supply of electricity while utilizing volatile renewableenergy, to work towards achieving sustainable data centers.
1.50

1.35

Chimneyp-PUE
Average

1.45
1.40

Racks
Building-based data centerAllOutsideAir18 mAllOutsideAir

Circulation

Containers

Mixed

1.30
1.25

9m

1.20
1.15
1.10

2.3 m

1.05

Container-based data center

1.00
2011.456789101112 2012.123(Year/Month)Figure 1: IZmo pPUE Performance*Patent pending *Proof-of-concept tests in planning stages

Figure 2: Chimney Data Centers (Concept Art)

Author:Isao Kubo
Data Center Service Department, Service Operation Division, IIJ39