Multi-layer data center cooling infrastructure

Embodiments are disclosed of a cooling system for use in a data center. The cooling system includes an IT region, a ceiling region, and a cooling air plenum sandwiched between the IT region and the ceiling region. The IT region includes one or more IT plenums that are coupled to the cooling air plenum to supply cooling air to the IT region, which can have house a plurality of IT racks that are clustered around the IT plenums and are adapted to house one or more pieces of liquid-cooled or hybrid-cooled information technology (IT) equipment. The ceiling region includes one or more ceiling plenums and one or more sets of heat exchangers, each heat exchanger being cooled by the cooling air delivered to the ceiling plenums. The cooling air plenum is fluidly coupled by a flow control to the one or more IT plenums and is fluidly coupled by a flow control to the one or more ceiling plenums or to the volume of the ceiling region between ceiling plenums.

TECHNICAL FIELD

The disclosed embodiments relate generally to liquid cooling systems for temperature control of electronic equipment and in particular, but not exclusively, to a multi-layer data center cooling system for temperature control in data center equipment.

BACKGROUND

Much modern information technology (IT) equipment such as servers, blade servers, routers, edge servers, etc., generates a substantial amount of heat during operation. The heat generated by individual components, especially high-power components such as processors, makes many of these individual components impossible or difficult to cool effectively with air cooling systems. Much modern IT equipment therefore requires liquid cooling or liquid-air hybrid cooling.

In liquid-air hybrid cooling, a liquid cooling loop is formed between heat-generating components in a piece of IT equipment and a liquid-air heat exchanger. Operation of the liquid-cooling loop has several steps. First, heat from the heat-generating components is transferred through fluid connections in the loop to a working liquid. The heated working liquid is then transferred to the liquid-to-air heat exchanger, in some cases through multiple heat transfer loops, where heat from the working liquid is transferred to air, usually atmospheric air. The working liquid, now cool after its heat has been transferred to the air, is returned to the heat-generating component to cool it again; the heated air is then exhausted into the atmosphere or used for some other purpose.

Because substantially all heat extracted from the components is eventually transferred to airflow, either directly or indirectly through a liquid cooling loop, efficient airflow management is needed for hybrid cooling, but existing airflow management solutions do not provide airflow efficiently enough to clusters of IT equipment, especially when the IT equipment is hybrid-cooled, meaning that the cooling air can be not efficiently or accurately distributed or managed to the heat load.

DETAILED DESCRIPTION

Embodiments are described of a multi-layer data center cooling system. Specific details are described to provide an understanding of the embodiments, but one skilled in the relevant art will recognize that the invention can be practiced without one or more of the described details or with other methods, components, materials, etc. In some instances, well-known structures, materials, or operations are not shown or described in detail but are nonetheless encompassed within the scope of the invention.

Reference throughout this specification to “one embodiment” or “an embodiment” means that a described feature, structure, or characteristic can be included in at least one described embodiment, so that appearances of “in one embodiment” or “in an embodiment” do not necessarily all refer to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

Airflow management can improve data center and IT cluster operating efficiency. With the increasing use of data centers, their energy consumption is becoming larger and larger—so much so that they can account for a large fraction of the total energy consumption of an entire city. There are therefore never-ending efforts to improve the energy efficiency of data centers. Modern data center designs require versatility and the ability to accommodate different generations of IT systems, but cost and efficiency always a consideration. This requires engineers and scientists to develop and invent new cooling architectures.

The described embodiments provide an innovative airflow distribution system using a multi-layer airflow distribution system that aims to improve airflow management efficiency and system versatility and flexibility. The described embodiments enable accurate and efficient distribution and management of airflow to the heat loads—both the heat loads that are directly air-cooled and the heat loads that are liquid-cooled. This not only separates the different cooling airflows but also enables delivery of airflow where is needed, since the air-cooled portion and liquid-cooled portion are different. The current architecture also compatible for different cooling solutions, and it can be easily implemented in either brownfield and greenfield data centers. In addition, it is a very efficient architectural concept for modular and container design.

Embodiments of an airflow distribution system using a multilayer design are described. The cooling airflow is distributed from an intermediate section of the system through its dedicated cooling air plenum. The airflow is distributed to both a ceiling region and an IT region to cool the liquid cooling unit and the IT equipment directly. The ceiling region is designed with raised air flow distribution, and the IT region is designed with cooling air blown down from the top. The cooling air plenum is the only portion receiving air from a cool air source, and the cool air is managed so that it can be distributed to different regions. The design also includes different cooling air intake modes using the cooling air plenum—e.g., one economized intake mode and an enhanced intake mode—to enable a more efficient system operation. The described embodiments of the system include several modules; this modular design enables the system to be accommodated in either a hyperscale data center design or a container design.

FIG. 1is a block diagram illustrating a side view of an embodiment of an electronic rack. In one embodiment, electronic rack100includes CDU101, rack management unit (RMU)102, and one or more server blades103A-103D, collectively referred to as server blades103. Server blades103can be inserted into an array of server slots respectively from front end104of electronic rack100. Note that although only four server blades103A-103D are shown, more or fewer server blades can be maintained within electronic rack100. Also note that the particular positions of CDU101, CMU102, and server blades103are shown for the purpose of illustration only; other arrangements or configurations of CDU101, CMU102, and server blades103can also be implemented. Further, the front door disposed on front end104and the back door disposed on back end105are optional. In some embodiments, there can no door on front end104and/or back end105.

In one embodiment, CDU101includes heat exchanger111, liquid pump112, and pump controller110. Heat exchanger111can be a liquid-to-liquid heat exchanger. Heat exchanger111includes a first tube having a first pair of liquid connectors coupled to external liquid supply/return lines131-132to form a primary loop, where the connectors coupled to the external liquid supply/return lines131-132can be disposed or mounted on back end105of electronic rack100. In addition, heat exchanger111further includes a second tube having a second pair of liquid connectors coupled to liquid manifold125, which can include a supply manifold to supply cooling liquid to server blades103and a return manifold to return warmer liquid back to CDU101. The processors can be mounted on the cold plates, where the cold plates include a liquid distribution channel embedded therein to receive the cooling liquid from the liquid manifold125and to return the cooling liquid carrying the heat exchanged from the processors back to the liquid manifold125. Supply/return lines131-132can be fluidly coupled to a liquid-to-air heat exchanger in a ceiling region (seeFIG. 2Aet seq.).

Each server blade103can include one or more IT components (e.g., CPUs, GPUs, memory, and/or storage devices). Each IT component can perform data processing tasks, where the IT component can include software installed in a storage device, loaded into the memory, and executed by one or more processors to perform the data processing tasks. Server blades103can include a host server (referred to as a host node) coupled to one or more compute servers (also referred to as compute nodes). The host server (having one or more CPUs) typically interfaces with clients over a network (e.g., Internet) to receive a request for a particular service such as storage services (e.g., cloud-based storage services such as backup and/or restoration), executing an application to perform certain operations (e.g., image processing, deep data learning algorithms or modeling, etc., as a part of a software-as-a-service or SaaS platform). In response to the request, the host server distributes the tasks to one or more of the compute servers (having one or more GPUs) managed by the host server. The compute servers perform the actual tasks, which can generate heat during the operations.

Electronic rack100further includes RMU102configured to provide and manage power supplied to server blades103and CDU101. RMU102can be coupled to a power supply unit (not shown) to manage the power consumption of the power supply unit, as well as other thermal management of the power supply unit (e.g., cooling fans). The power supply unit can include the necessary circuitry (e.g., an alternating current (AC) to direct current (DC) or DC to DC power converter, battery, transformer, or regulator, etc.,) to provide power to the rest of the components of electronic rack100.

In one embodiment, RMU102includes optimal control logic111and rack management controller (RMC)122. The optimal control logic111is coupled to at least some of server blades103to receive operating status of each of the server blades103, such as processor temperatures of the processors, the current pump speed of the liquid pump112, and liquid temperature of the cooling liquid, etc. Based on this information, optimal control logic111determines an optimal pump speed of the liquid pump112by optimizing a predetermined objective function, such that the output of the objective function reaches the maximum while a set of predetermined constraints is satisfied. Based on the optimal pump speed, RMC122is configured to send a signal to pump controller110to control the pump speed of liquid pump112based on the optimal pump speed.

FIGS. 2A-2Btogether illustrate an embodiment of a multi-layer data center cooling system200.FIG. 2Ais a view looking from the ends or rows of racks, whileFIG. 2Bis a view of the same racks viewed from the side, either from the cold aisle looking at the front side of the racks, or from the hot aisle looking at the rear side of the racks. Cooling system200is positioned in a data center201and includes three layers: an information technology (IT) layer202in which IT equipment requiring cooling is placed, a ceiling layer204positioned above the IT layer, and a cooling plenum layer206sandwiched between the IT layer and the ceiling layer.

IT layer202is the layer where IT equipment requiring cooling is placed. One or more pieces of IT equipment such as servers, blade servers, routers, etc., are positioned in one or more IT racks208. Racks208can also include liquid-cooling elements that are fluidly coupled to liquid cooling elements of the IT equipment housed in the rack (see, e.g.,FIG. 1), as well as being fluidly coupled to liquid cooling elements of the data center's cooling system, as further discussed below. Examples of liquid-cooling elements that can be placed in rack208include heat exchangers, pumps, pump controllers and associated logic, manifolds, and fluid connections to provide the fluid coupling among elements (seeFIG. 1). Each rack208can also include provisions for cooling of the IT equipment using cool air flowing through the rack. Each rack208has a front side through which cool air can enter the rack and a rear side through which hot air can exit the rack. Some embodiments of racks208, or of the IT equipment inside the rack, can use air-moving elements such as fans to draw cool air through the front side and exhaust hot air out the backside.

In IT layer202, one or more racks208are arranged in rows and clustered (i.e., grouped into clusters) around an IT plenum210. In the illustrated embodiment IT plenums210are vertical, but in other embodiments they can have a different orientation. In each cluster, each rack208has its front side fluidly coupled to a corresponding IT plenum210, so that cool air can flow from each IT plenum210into the front ends of all the racks208that are clustered around it. The rear side of each rack208is fluidly coupled to the interior volume211of the IT layer, such that the racks and up grouped into a hot aisle/cold aisle arrangement, with IT plenums210forming a cold aisle and volume211forming a hot aisle. In the illustrated embodiment, IT layer202also includes exhaust vents212that fluidly couple the interior volume211of the IT layer to the outside of the data center. Hot air exiting the rear side of each rack can be exhausted or vented from interior volume211to the exterior of data center201through exhaust vents212.

Ceiling layer204is positioned above IT layer202and includes a volume214within which are positioned one or more liquid-to-air heat exchangers216—that is, in this design, heat exchangers in which heat is transferred from a liquid to air or, put differently, heat exchanges in which air is used to cool a liquid. Each heat exchanger216is fluidly coupled to at least one rack's supply and return connections (seeFIG. 1, elements132and131) by a pair of fluid connections218. Each pair of fluid connections218includes a supply connection and a return connection, as indicated by the arrows. The illustrated embodiment of cooling system200has a one-to-one correspondence between racks208and heat exchangers216—i.e., each rack is fluidly coupled to a single corresponding heat exchanger. But in other embodiments there can be a one-to-many correspondence between racks and heat exchangers (each rack is fluidly coupled to more than one heat exchanger), a many-to-one correspondence between racks and heat exchangers (more than one rack is fluidly coupled to each heat exchanger), or a many-to-many correspondence between racks and heat exchangers (more than one rack is coupled to more than one heat exchanger). Also positioned within volume214of ceiling layer204are ceiling plenums220. In ceiling layer204, one or more liquid-to-air heat exchangers216are clustered (i.e., grouped into clusters) around each ceiling plenum220. In each cluster, each heat exchanger216is fluidly coupled to its corresponding ceiling plenum220(this can be understood as hot air containment), so that hot air can flow from each heat exchanger into its corresponding ceiling plenum; in this embodiment, then, ceiling plenums220act as exhaust plenums, which means the hot air is exhausted directly to the ambient. In other embodiments, ceiling plenums220can be implemented in different locations or sides of ceiling layer204.

Cooling plenum layer206is sandwiched between IT layer202and ceiling layer204to provide cooling air to both layers. Cooling plenum layer206includes at least one air intake226through which cool air can enter the layer. In the illustrated embodiment, a pair of air intakes226are positioned in the walls of the data center, so that cool outdoor air can be brought into the cooling air plenum from outside data center201. Cooling plenum layer206also includes multiple outlets through which cool air from the cooling plenum can be directed and distributed into the IT layer and the ceiling layer. In the illustrated embodiment, the cooling plenum layer206is fluidly coupled to IT plenums210by flow controls222, and is also fluidly coupled to volume214of ceiling layer204by flow controls224. Flow controls222and224can be used to regulate the flow into their respective plenums, and can range from completely open to completely closed and anywhere in between. In one embodiment, flow controls222and224can be adjustable louvers that control the amount of airflow passing through them, but in other embodiments they can be another type of flow control. For instance, in other embodiments the flow controls can be perforated openings or any similar structures. Although not shown in this figure, cooling plenum layer206can also include air-moving elements such as fans to assist or optimize airflow through the cooling plenum layer (see, e.g.,FIG. 4).

In operation of system200, cool air flows into cooling plenum layer206through intakes226. The airflow path through the system is represented in the figures by dashed arrows. Cool air flowing into and through the cooling plenum layer is then supplied to the IT layer and the ceiling layer. In the IT layer, cool air from the cooling plenum layer flows through flow controls222into IT plenums210. The front sides of racks208are fluidly coupled to the IT plenums, so that the cool air flowing from the cooling air plenum into the IT plenums is directed into racks208. Cool air flowing into the front side of each rack208flows over the IT components within the rack, absorbs heat from the IT components, and exits the rear side of each rack. The hot air exiting the back of each rack208into IT layer volume211is then exhausted through vents212to the outside of the data center.

In ceiling layer204, the liquid cooling loops are in operation, with hot and cool liquid flowing through the supply and return lines218: hot liquid from racks208enters heat exchangers216and is cooled, and the cool fluid then flows back to the racks. Although not shown in the drawing, a pumping system for recirculating the fluid running with218can be present in some embodiments. But in other embodiments, no pump may be needed for the loop218, such as for a self-driving thermosiphon system. At the same time, cool air from cooling plenum layer206flows through flow controls224into ceiling volume214. Once in ceiling volume214, the cool air flows through heat exchangers216and into ceiling plenums220, which in this embodiment act as exhaust plenums. The air flowing through the heat exchangers extracts heat from the liquid as it flows through the heat exchanger. Having flowed through heat exchangers216, the now-hot air flows into ceiling plenums220and flows out of the data center.

FIG. 3illustrates another embodiment of a data center cooling system300. System300includes three layers—an IT layer302, a ceiling layer304positioned above the IT layer, and a cooling plenum layer306sandwiched between the IT layer and the ceiling layer. IT layer302and cooling plenum layer306are substantially similar to the IT layer202and cooling plenum layer206in system200, and include substantially the same elements arranged in substantially the same ways. The primary difference between cooling systems200and300is in ceiling layer304.

Ceiling layer304includes a volume308within which are positioned one or more liquid-to-air heat exchangers310. Each heat exchanger310is fluidly coupled to rack's supply and return connections (seeFIG. 1) by a pair of fluid connections218; each pair of fluid connections218includes a supply connection and a return connection, as indicated by the arrows. As in system200, in cooling system300there can be a one-to-one correspondence between racks208and heat exchangers216(each rack is fluidly coupled to a single corresponding heat exchanger), a one-to-many correspondence between racks and heat exchangers (each rack is fluidly coupled to more than one heat exchanger), a many-to-one correspondence between racks and heat exchangers (more than one rack is fluidly coupled to each heat exchanger), or a many-to-many correspondence between racks and heat exchangers (more than one rack is coupled to more than one heat exchanger).

Also positioned within volume214of ceiling layer204are ceiling plenums312that are fluidly coupled to cooling plenum layer306by flow controls314. In the illustrated embodiment ceiling plenums312are vertically oriented, but in other embodiments they can be oriented differently. In ceiling layer304, one or more liquid-to-air heat exchangers310are clustered (i.e., grouped into clusters) around each ceiling plenum312. In each cluster, each heat exchanger310is fluidly coupled to its corresponding ceiling plenum312, so that cold air can flow from each ceiling plenum into its corresponding heat exchanger310. In system300, then, ceiling layer304has a cold aisle/hot aisle arrangement, with ceiling plenums312forming cold aisles and volume308forming hot aisles, and this can be understood as that the ceiling layer forms a hot aisle, since the cold aisle is contained as plenums312.

In operation of system300, IT layer302and cooling plenum layer306operate similarly to IT layer202and cooling plenum layer202of system200, but ceiling layer304operates differently. In ceiling layer304, the liquid cooling loops are in operation, with hot and cool liquid flowing through the supply and return lines218. At the same time, cool air from cooling plenum layer306flows through flow controls314into ceiling plenums312. From ceiling plenums312, the cool air flows into and through heat exchangers216into exhaust volume308, extracting heat from the liquid as it flows through the liquid-to-air heat exchanger. The now-hot air flows from volume308through exhausts316to the exterior of data center201.

FIG. 4illustrates another embodiment of a data center cooling system400. System400is in most respects similar to system300: it includes an IT layer302, a ceiling layer304positioned above the IT layer, and a cooling plenum layer306sandwiched between the IT layer and the ceiling layer. All three layers are substantially as described above in connection withFIG. 3, except for ceiling layer304, which is modified as described below.

The primary difference between cooling systems300and400is the inclusion in system400of an external cooling module402. System400provides a more complete solution that combines both system300and cooling module for supplying airflow to the cooling plenum layer306. In some circumstances, the air quality outside data center201—measure, for instance, by temperature, humidity, and the presence of chemicals or particulates, or other quantities—may not allow ambient air to be used for cooling without further conditioning; in such circumstances, cooling module402can provide the conditioning. In the illustrated embodiment, cooling module402includes a cooling unit404to change the temperature of outside air, usually by cooling it; one or more filtration units406to provide chemical or mechanical filtration that removes chemicals and particulates; and a fan unit408to drive the cooled and/or filtered air into cooling plenum layer306. In one embodiment, the cooling unit404may be the same as cooling module402. Other embodiments of cooling module402can include more or less units than shown. Other components used to implement open or closed loop control of cooling module402—temperature sensors, controllers, etc.—are not shown in the figure but can nonetheless be used in some embodiments.

Cooling module402is fluidly coupled to cooling plenum layer306by a flow control410, so that the flow control can be used to regulate the amount of air entering the cooling plenum layer. Use of cooling module408allows more cubic feet per minute (CFM) of cool air to be circulated through the data center, thus providing more heat transfer capacity and allowing the data center to operate at higher power (i.e., more kilowatts (kW)). The single-intake design of cooling plenum layer306provides a convenient and accessible place for the addition of cooling module402, making the entire cooling system400more flexible.

In operation of system400, IT layer302and cooling plenum layer306operate substantially as described above for system300, except that the cool air flowing into cooling plenum layer306is now pre-conditioned (heated/cooled, humidified/dehumidified and filtered) and is driven through the plenum cooling layer by fan unit408. In addition to fan unit408, flow control410can help regulate or optimize the amount of cool air flowing into the cooling plenum layer. Hot air is exhausted from volume308of ceiling layer304as described above for system300, but in the ceiling layer, exhaust316is eliminated at the location where cooling module402is added, so that hot air is exhausted from the volume308through a different number of exhausts316than in system300.

FIG. 5illustrates another embodiment of a data center cooling system500. System500is in most respects similar to systems300and400: it includes an IT layer302, a ceiling layer304positioned above the IT layer, and a cooling plenum layer306sandwiched between the IT layer and the ceiling layer. All three layers are substantially as described above in connection withFIGS. 3-4, except for ceiling layer304, which is slightly modified as described below.

The primary difference between cooling system500and cooling systems300and400is the inclusion in system500of a recirculation unit502. In some circumstances, the air quality outside data center201—measured, for instance, by temperature, humidity, the presence of chemicals or particulates, or other quantities—may not allow the use of ambient air from the outside for cooling the data center. One way to avoid using outside air is to recirculate the air within the data center instead of taking in ambient air from the outside. Recirculation module502can provide that recirculation.

Recirculation module502is positioned outside data center201and includes a volume504. An external cooling unit506is positioned within volume504. Volume308of ceiling layer304is fluidly coupled to volume504via exhaust316, while volume211of IT layer302is coupled to volume504by exhaust212. External cooling unit506is also fluidly coupled to the exterior, directly or via volume504, so that outside air can flow through external cooling unit. External cooling unit506is also fluidly coupled to cooling plenum layer306by a fan unit508and a flow control510. In various embodiments, the external cooling unit can be a chilled water system, an evaporative cooling unit, or any other type of heat exchanger for extracting the heat from the internal air. And, although illustrated as a single unit, in other embodiments the cooling can be performed by multiple units. Other components used to implement open or closed loop control of recirculation unit502—temperature sensors, controllers, etc.—are not shown in the figure but can nonetheless be present in some embodiments.

In operation of system500, IT layer302and ceiling layer304operate substantially as before: cool air flows from cooling plenum layer306into IT layer302through IT plenums210and also flows into ceiling layer304through ceiling plenums312. Hot air from both the ceiling layer and the IT layer is directed into recirculation unit502through exhausts316and212respectively. After entering the recirculation unit, the hot air from IT layer302and ceiling layer304is directed into external cooling unit506, where heat from the hot internal air is transferred to the external air. In some embodiments air ducting may be used in recirculation unit502for containing air flow between the IT and ceiling layers and the external cooling unit506, so that outside air is separated from internal air. After transferring its heat to the external air, the now-cooled air that originated from IT layer302and ceiling layer304is recirculated into cooling plenum layer306by fan unit508and flow control510, which then again carries the cool air to IT layer302and ceiling layer304.

FIG. 6illustrates another embodiment of a data center cooling system600. System600is in most respects similar to system300: it includes an IT layer602, a ceiling layer604positioned above the IT layer, and a cooling plenum layer606sandwiched between the IT layer and the ceiling layer. IT layer602and ceiling layer604are substantially as described above in connection withFIGS. 3-5, except for ceiling layer604, which is slightly modified as described below. The primary difference between cooling system600and cooling system300is that in system600, cooling plenum layer606has multiple air intakes with different functions. In the illustrated embodiment, cooling plenum layer has two air intakes: an enhanced intake608and an economized intake610.

In some circumstances, the air quality outside data center201—measured, for instance, by temperature, humidity, the presence of chemicals or particulates, or other quantities—may not allow the use of ambient air taken directly from outside for cooling the data center. In such situations, it can be desirable to enhance the ambient air from the outside. Enhanced air intake608, as its name implies, provides enhancement functions. In the illustrated embodiment, enhanced air intake608includes a cooling system with an evaporator612, a condenser614, and a fan616. Evaporator612is positioned at or near the entrance of cooling plenum layer606, so that air entering the cooling plenum layer through enhanced intake608flows through the evaporator and is cooled before proceeding into the rest of the system. Condenser614is positioned outside data center201and is fluidly coupled to evaporator612by a refrigerant loop615that includes a return line through which vapor or a liquid/vapor mix can travel through from the evaporator to the condenser, and a supply line through which liquid can travel from the condenser to the evaporator. Although not shown in this figure, the refrigerant loop can also include a compressor. A fan616can be coupled to condenser614to help it condense vapor into liquid more quickly. In addition to the cooling system, enhanced intake608can include a flow control621that can completely close enhanced intake610, completely open the intake, or adjust it anywhere in between completely closed and completely open. Enhanced intake608can also include one or more mechanical or chemical filters618.

Economized intake610can include a fan622to increase the flow of air into cooling plenum payer606, as well as a flow control624to adjust the amount of air flowing into the plenum. The flow control can completely close economized intake610, can completely open the intake, or can adjusted anywhere in between completely closed and completely open. Economized intake610can also include one or more mechanical or chemical filters625.

In operation, enhanced intake608and economized intake610can operate independently. At any given time one can be open and the other closed, or both can be open simultaneously. When open simultaneously, enhanced intake608and economized intake610can have the same or different flow rates. Once cool airflow is established using one or both intakes, the cool air proceeds through cooling plenum layer606to IT layer602and ceiling layer604, as discussed above for system300. System600provides flexibility to accommodate different heat load requirements and different ambient conditions.

FIG. 7illustrates another embodiment of a data center cooling system700. System700is in most respects similar to system600: it includes an IT layer702, a ceiling layer704positioned above the IT layer, and a cooling plenum layer706sandwiched between the IT layer and the ceiling layer. Ceiling layer704is substantially the same as ceiling layer604, as described above forFIG. 6, but IT layer702and cooling plenum layer706differ from their respective counterparts in system600—IT layer602and cooling plenum layer606, respectively.

Cooling plenum layer706shares some similarities with cooling plenum layer606in that it has multiple air intakes with different functions. In the illustrated embodiment, cooling plenum layer has an enhanced intake608and an economized intake610, both of which can have all the same components and attributes described above inFIG. 6for these same elements. The primary difference between cooling plenum layers606and706is that cooling plenum layer706is divided into two parts: a lower cooling plenum7061and an upper cooling plenum706u. Lower cooling plenum7061is fluidly coupled to enhanced intake608and to IT plenums210, so that it supplies cool air only to IT layer702. Upper cooling plenum706uis fluidly coupled to economized intake610and to ceiling plenums312, so that it supplies ambient air only to ceiling layer704. System700can be useful, for instance, when the ceiling layer and IT layer have different cooling requirements. And even though there are two plenums7061and706u, in other embodiments they can be combined to function as a single channel.

IT layer702also shares some similarities with IT layer602, in that it houses racks208that are clustered around IT plenums210. But IT layer702also includes a return plenum708. The illustrated embodiment has one return plenum, but other embodiments can have more than one. Return plenum708returns hot air from IT layer702to the enhanced intake608, where it can be cooled, filtered, and directed back into lower plenum7061. IT layer702also differs from IT layer602in that its IT plenums210are fluidly coupled, via flow controls, only to the lower cooling plenum7061instead of being coupled to the entire cooling plenum layer. In one embodiment, if direct outside air is taken to the plenum7061and then supplied to the210, exhaust212is used for exhausting hot air.

In operation, enhanced intake608and economized intake610can operate independently. At any given time one can be open and the other closed, or both can be open simultaneously. When open simultaneously, enhanced intake608and economized intake610can have the same or different flow rates. Once cool airflow is established using one or both intakes, the cool air proceeds through cooling plenum layer7061to IT layer702and through cooling plenum layer706uto ceiling layer704, after which the system operates substantially as discussed above for system300. System700provides flexibility to accommodate different heat load requirements and different ambient conditions.

Other embodiments of cooling systems are possible besides the ones described above. For instance:The current architecture can be coupled with different types of cooling solutions, or cooling loops;The multilayer air distribution system can be extended for even more layers for airflow distributions; orDifferent IT room cold aisle containment and hot aisle containment can be accommodated in the present architecture.

The above description of embodiments is not intended to be exhaustive or to limit the invention to the described forms. Specific embodiments of, and examples for, the invention are described herein for illustrative purposes, but various modifications are possible.