Hybrid data center rack

Embodiments are disclosed of an information technology (IT) rack. The IT rack includes an equipment enclosure with a front, a rear, and one or more partitions, each partition adapted to receive one or more pieces of liquid-cooled information technology (IT) equipment. One or more cooling doors are positioned on the back of the equipment enclosure, each having therein a heat exchanger, the heat exchanger that is fluidly coupled to at least one of the one or more pieces of liquid-cooled IT equipment forming liquid cooling loops. Each of the one or more cooling doors is movable between a first position where the cooling door extends across the back of at least one partition so that air from the interior of the at least one partition flows through the cooling door, and a second position where air from outside the equipment enclosure flows through the cooling door. Fans are used in the middle section of rack or between two enclosures for assisting airflow management. Dedicated space for the fans are designed on the rack.

TECHNICAL FIELD

The disclosed embodiments relate generally to liquid cooling systems for temperature control of electronic equipment and in particular, but not exclusively, to a hybrid data center rack for temperature control in data center equipment.

BACKGROUND

Much modern information technology (IT) equipment such as servers, blade servers, routers, edge servers, etc., generates a substantial amount of heat during operation. The heat generated by individual components, especially high-power components such as processors, makes many of these individual components impossible or difficult to cool effectively with air cooling systems. Much modern IT equipment therefore requires liquid cooling or liquid-air hybrid cooling.

As a result of the requirement for liquid cooling, some pieces of IT equipment have an on-board cooling system that is thermally coupled to individual components that need cooling. But these on-board cooling systems usually do not operate in isolation. They are usually coupled to at least one larger cooling system, such as liquid cooling system in an electronics rack. The rack's cooling system can also be coupled to the liquid cooling system of a larger facility such as a data center. In such a system, the data center's cooling system circulates a working fluid through the rack cooling system, which in turn circulates the working fluid through the cooling system on the piece of IT equipment.

One challenge in designing data centers and data center racks is the mismatch between the lifetime of the data center and the lifetime of IT equipment housed in the data center. The data center and its facilities, electrical systems, cooling systems, etc., change much more slowly that the electronics housed within. Generally, the electronics change quickly and become more customized including their form factors, packaging method, system design, mechanical/thermal (air cooling and liquid cooling)/structural solutions. This rapid change leads to several associated problems. For example, different operating conditions can lead to different rack power in different scenarios. Data center design and corresponding cooling and power source availabilities might not be able to keep up—e.g., some data centers provide only cooling air, which might not be enough considering the rising power and heating, and variations in the cooling design in the IT equipment.

DETAILED DESCRIPTION

Embodiments are described of IT racks having one or more cooling doors. Specific details are described to provide an understanding of the embodiments, but one skilled in the relevant art will recognize that the invention can be practiced without one or more of the described details or with other methods, components, materials, etc. In some instances, well-known structures, materials, or operations are not shown or described in detail but are nonetheless encompassed within the scope of the invention.

Reference throughout this specification to “one embodiment” or “an embodiment” means that a described feature, structure, or characteristic can be included in at least one described embodiment, so that appearances of “in one embodiment” or “in an embodiment” do not necessarily all refer to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

The disclosed embodiments of IT racks provide high resilience and compatible hardware platforms for deploying different types of servers in different data center environments. In the described racks, one or more sections of cooling doors equipped with heat exchanger coils are attached to the rear side of the rack. The cooling doors can be operated in either first (closed) position (extending across the rear side of the rack) or in a second (open) position for expanding the cooling capabilities. Fan units for moving cooling airflow can be used in the middle sections of the rack or integrated between two adjacent racks. If a fan unit is used in a single rack, the rack can have a dedicated fan section to house the fan unit. Containment solutions are used in arranging the racks, and several system layouts for a data center containment and the rack architectures are described.

Features of this advanced hybrid rack configuration and design include:Rear door heat exchanger for extending heat spreading.Optional integrated fan unit for airflow management.Cooling adjustment for non-uniform server power population.Support for operating and cooling in both air-cooled data centers and liquid-cooled data centers, as well as air cooled IT equipment and liquid cooled IT equipment.Enabling different data center system level integrate designs.Expanding the role of an IT rack beyond performing as a cabinet. The rack can be a solution for deploying different types of servers in different type of data centers, at the same time, supporting for different type of cooling requirements and providing elastic cooling capacity expanding capability.

FIG.1is a block diagram illustrating a side view of an embodiment of an electronic rack. In one embodiment, electronic rack100includes CDU101, rack management unit (RMU)102, and one or more server blades103A-103D, collectively referred to as server blades103. Server blades103can be inserted into an array of server slots respectively from front end104of electronic rack100. Note that although only four server blades103A-103D are shown, more or fewer server blades can be maintained within electronic rack100. Also note that the particular positions of CDU101, CMU102, and server blades103are shown for the purpose of illustration only; other arrangements or configurations of CDU101, CMU102, and server blades103can also be implemented. Further, the front door disposed on front end104and the back door disposed on back end105are optional. In some embodiments, there can no door on front end104and/or back end105.

In one embodiment, CDU101includes heat exchanger111, liquid pump112, and pump controller110. Heat exchanger111can be a liquid-to-liquid heat exchanger. Heat exchanger111includes a first tube having a first pair of liquid connectors coupled to external liquid supply/return lines131-132to form a primary loop, where the connectors coupled to the external liquid supply/return lines131-132can be disposed or mounted on back end105of electronic rack100. In addition, heat exchanger111further includes a second tube having a second pair of liquid connectors coupled to liquid manifold125, which can include a supply manifold to supply cooling liquid to server blades103and a return manifold to return warmer liquid back to CDU101. The processors can be mounted on the cold plates, where the cold plates include a liquid distribution channel embedded therein to receive the cooling liquid from the liquid manifold125and to return the cooling liquid carrying the heat exchanged from the processors back to the liquid manifold125. Supply/return lines131-132can be fluidly coupled to a cooling door (seeFIG.2Aet seq.), to data center liquid cooling system, or to both.

Each server blade103can include one or more IT components (e.g., CPUs, GPUs, memory, and/or storage devices). Each IT component can perform data processing tasks, where the IT component can include software installed in a storage device, loaded into the memory, and executed by one or more processors to perform the data processing tasks. Server blades103can include a host server (referred to as a host node) coupled to one or more compute servers (also referred to as compute nodes). The host server (having one or more CPUs) typically interfaces with clients over a network (e.g., Internet) to receive a request for a particular service such as storage services (e.g., cloud-based storage services such as backup and/or restoration), executing an application to perform certain operations (e.g., image processing, deep data learning algorithms or modeling, etc., as a part of a software-as-a-service or SaaS platform). In response to the request, the host server distributes the tasks to one or more of the compute servers (having one or more GPUs) managed by the host server. The compute servers perform the actual tasks, which can generate heat during the operations.

Electronic rack100further includes RMU102configured to provide and manage power supplied to server blades103and CDU101. RMU102can be coupled to a power supply unit (not shown) to manage the power consumption of the power supply unit, as well as other thermal management of the power supply unit (e.g., cooling fans). The power supply unit can include the necessary circuitry (e.g., an alternating current (AC) to direct current (DC) or DC to DC power converter, battery, transformer, or regulator, etc.,) to provide power to the rest of the components of electronic rack100.

In one embodiment, RMU102includes optimal control logic111and rack management controller (RMC)122. The optimal control logic111is coupled to at least some of server blades103to receive operating status of each of the server blades103, such as processor temperatures of the processors, the current pump speed of the liquid pump112, and liquid temperature of the cooling liquid, etc. Based on this information, optimal control logic111determines an optimal pump speed of the liquid pump112by optimizing a predetermined objective function, such that the output of the objective function reaches the maximum while a set of predetermined constraints is satisfied. Based on the optimal pump speed, RMC122is configured to send a signal to pump controller110to control the pump speed of liquid pump112based on the optimal pump speed.

FIGS.2A-2Ctogether illustrate an embodiment of an information technology (IT) rack200.FIGS.2A-2Bare top cross-sections of the rack. IT rack200has a front and a rear and includes an equipment enclosure202divided into one or more vertically-oriented partitions204. This means the rack can not only be arranged in U space along the height, it also divided into several regions across its width direction. In the illustrated embodiment, equipment enclosure202can be a standard IT rack that has been divided into two partitions204aand204b, but in other embodiments the equipment enclosure need not be a standard IT rack and can have more or less partitions than shown. Each partition204is an IT region adapted to receive one or more pieces of liquid-cooled IT hardware, such as servers, routers, cooling components, and the like, as shown for instance inFIG.1. Each piece of IT hardware can, for example, include a chassis206within which are housed electronics and liquid cooling components208. The server chassis and fan can be populated to the rack at its dedicated space from the front side of the rack.

A cooling door210is coupled to enclosure202by one or more hinges212positioned on the rear of the enclosure. Cooling door210has a heat exchanger therein that is fluidly coupled to the liquid-cooled IT hardware positioned in partitions204aand204b, forming a loop with the liquid cooling components208. The heat exchanger rotates together with the door. Cooling door210can rotate about hinges212from a first position extending cross the rear of the enclosure to a second position where the cooling door is at a non-zero angle β relative to its first position (seeFIG.2B). Put differently, cooling door210is in its first position when β=0 and is in its second position when β≠0. The cooling door's first position can also be described as “closed” and its second position as “open.” In the illustrated embodiment, in the open position angle β is substantially 180 degrees, but in other embodiments angle β can be any angle between zero degrees and 270 degrees. Other embodiments of rack200can have cooling doors that move between their first and second positions other than by rotation about a hinge (see, e.g.,FIGS.5A-5B). In addition, if the cooling door is connected to a cooling source such as an external cooling loop, the external fluid supply and return can be coupled at the hinge or near the location of the hinge. An embodiment of a cooling door210is described below in connection withFIG.2C.

In the illustrated embodiment a fan unit214is positioned in a fan region between partitions204aand204b. In embodiments with more than two partitions, a fan unit can be placed between each pair of partitions. In the illustrated embodiment fan unit214pulls in or entrains air from the sides of rack202and exhausts it out the rear of the enclosure (seeFIG.2B). In one embodiment, fan unit214can include one or more radial fans that guide airflow to cover as much area as possible in each partition before leaving the rack. In one embodiment, each partition can be equipped with individual fan/fans for airflow management. The fan unit is an option and can be used in certain high density application scenarios, performance and airflow management optimization, and so on. Other embodiments can omit the fan unit between any pair of partitions, and embodiments that omit the fan unit can also omit the fan region in which the fan unit is installed.

FIG.2Bshows rack200in an operating mode, in which cooling door210is in its second (open) position, with β set to 180 degrees. Airflow is also shown in the figure by dashed arrows, which show the air flow paths when the cooling door is used as a closed loop. Airflow through cooling door210is supplied by the room cooling air unit of the data center room in which rack200is located. One portion of the cooling airflow passes through cooling door, and other portion of airflow passes through both sides of the rack before it exits the rear of the rack. Rack200can also operate in other modes. In one operating mode, the cooling door remains closed (β=0), in which case the air flowing through the cooling door and the heat exchanger is primarily air exiting the rear of the equipment enclosure. In another operating mode, the cooling door can be open, but at an angle less than 90 degrees (0≤β≤90), in which case the air flowing through the cooling door may be a mix of facility air and exiting enclosure air. In still another operating mode the cooling door can be open, but at any angle greater than or equal to 90 degrees (β≥90), depending on actual use cases and data center room/IT cluster layout and rack arrangement configurations.

FIG.2Cillustrates an embodiment of a cooling door210. Cooling door210includes a frame216which includes two protrusions218at the top and bottom of one side. Protrusions218include provisions for accommodating hinges212. A heat exchanger is positioned within frame216and can be fluidly coupled to one or more liquid cooling systems of IT equipment within the partitions of the rack to which cooling door210is coupled (seeFIGS.2A-2B). In a closed-loop configuration, the heat exchanger can be coupled only to the liquid cooling systems of electronics in the enclosure, thus forming a cooling loop that is self-contained in the rack—that is, a closed cooling loop. In an open-loop configuration, the heat exchanger can, instead or in addition, be coupled to a data center liquid cooling loop (i.e., a cooling loop external to the rack), so that liquid cooling is no longer self-contained within the rack.

In the illustrated embodiment the heat exchanger is a liquid-to-air heat exchanger including a tube220thermally coupled to fins222to enhance heat transfer from liquid flowing through tube220into air flowing over tube220and fins222. In one embodiment, the front and back of the cooling door can be left completely open to allow unimpeded flow of air through the door and the heat exchanger, as illustrated inFIG.2B. In other embodiments, flow-through elements such as screens can be placed at the front, back, or both front and back, of the cooling door to protect elements within the door while allowing airflow through the door and the heat exchanger. In other embodiments, tube arrangement layouts can be different, such as in a parallel manner.

Hinge212does not translate, or translates minimally, during movement of the cooling door from its first position to its second position, so that in the illustrated embodiment, tube220of the heat exchanger can be coupled to the liquid cooling systems of IT equipment within the partitions through hinge212, so that it forms a closed liquid-cooling loop within rack200. In one embodiment, for instance, enclosure202can include a fluid reservoir or manifold (not shown) to which the cooling systems are coupled, and the reservoir, pump or manifold can then be fluidly coupled to the heat exchanger through hinge212. In another embodiment, cooling door210can include a fluid reservoir or manifold (not shown) that can then be coupled to the liquid cooling systems in the rack through the hinge. In other embodiments of cooling door210, the fluid connections between the heat exchanger and the liquid cooling systems need not be through the hinge. For instance, the fluid connection can be accomplished using flexible hoses fluidly connected directly to the heat exchanger, or through a flexible hose fluidly coupled between a fluid reservoir or a manifold and the heat exchanger. In another embodiment, the heat exchanger in cooling door210can be fluidly coupled to an external liquid-cooling loop, for instance a liquid-cooling loop in a data center where the rack is located. Connection to an external liquid-cooling loop can be in addition to, or instead of, the closed-loop connection described above. In another embodiment in a closed loop design, the rack distribution manifold and the hinge may be combined as a one part. Either embodiment forms a different cooling architecture and might require certain additional equipment, but the configuration of rack200itself is compatible with these applications scenarios without any modification.

FIGS.3A-3Btogether illustrate another embodiment of an IT rack300.FIG.3Ashows rack300with cooling doors in their first position,FIG.3Bshows the cooling doors in their second position. IT rack300is in most respects similar to IT rack200: it has a front and a rear and includes an equipment enclosure202divided into vertically-oriented partitions204aand204b, but other embodiments can have more or less partitions than shown. Each partition204is an IT region adapted to receive one or more pieces of liquid-cooled IT hardware, such as servers, routers, cooling components, and the like, as shown for instance inFIG.1. Each piece of IT hardware can, for example, include a chassis206within which are housed electronics and liquid cooling components208. In one embodiment a fan region with a fan unit214is positioned in a fan region between partitions204aand204b, although other embodiments need not include the fan unit, and embodiments without fan unit214need not include a fan region to house the fan unit.

The primary difference between racks200and300is that rack300includes multiple cooling doors210. In the illustrated embodiment, rack300includes two cooling doors210aand210b, each of which extends across a single partition when in its first (closed) position. This is in contrast to rack200, in which a single cooling door extends across multiple partitions when in its first position. In one embodiment, cooling doors210aand210bhave the construction described above in connection withFIG.2C, but in other embodiments the can have a different construction than shown and in still other embodiments both cooling doors need not have the same construction. Rack300has all the same operational modes as racks200.

FIG.3Billustrates rack300with cooling doors210in a second (open) position. Door210arotates about hinge212athrough an angle βa from its first position, shown inFIG.3A, to its second position. Similarly, cooling door210brotates about hinge212bthrough an angle βb to its second position. In the illustrated embodiment βa is different than βb (βa≠βb), but in other embodiments βa can be the same as βb (βa=βb). In one embodiment, for instance, βa=βb=180 degrees (see, e.g.,FIGS.6-9), but each of βa and βb can take any value between 0 degrees and 270 degrees. The fan214can be a single space in the middle between204aand204bas shown inFIG.3A, or two separate spaces as shown inFIG.3B.

FIG.4illustrates an embodiment of an IT rack400, which can also be understood as rack system). Rack400has a front and a rear and includes two equipment enclosures402and404, each of which is divided into vertically-oriented partitions: enclosure402is divided into partitions204aand204band enclosure404is divided into partitions404aand404b. Other embodiments of rack can have more or less enclosures than shown, each enclosure can have more or less partitions than shown, and each enclosure need not have the same number of partitions. In one embodiment each enclosure402and404is a standard IT rack, but in other embodiments that need not be the case. A difference between rack400and rack300, then, is that rack300is a single enclosure divided into one or more partitions, while rack400combines multiple enclosures, each of which can be divided into one or more partitions. As in racks200and300, each partition402a-402band404a-404bis an IT region adapted to receive one or more pieces of liquid-cooled IT hardware, such as servers, routers, cooling components, and the like, as shown for instance inFIG.1. Each piece of IT hardware can, for example, include a chassis206within which are housed electronics and liquid cooling components208. Rack400has all the same operational modes as racks200and300.

A pair of cooling doors410are positioned across at least part of the rear of rack400: cooling door410ais rotatably coupled to enclosure402by hinge412aand in its first (closed) extends across enclosure402and its partitions, while cooling door410bis rotatably coupled to enclosure404by hinge412band in its first (closed) position extends across enclosure404and its partitions. Cooling doors410a-410bcan rotate about hinges412from a first position across the rear of the enclosure to a second (open) position where the cooling door is at a non-zero angle β relative to its first position. Cooling door410arotates about hinge412athrough an angle βa from its first position to its second position and cooling door410brotates about hinge412bthrough an angle βb to its second position. In the illustrated embodiment βa is different than βb (βa≠βb), but in other embodiments pa can be the same as βb (βa=βb) (see, e.g.,FIGS.6-9). In the illustrated embodiment, for instance, βa=180 degrees and βb=0 degrees, but each of βa and βb can take any value between 0 degrees and 270 degrees. Cooling doors410aand410bhave heat exchangers therein that are fluidly coupled to the liquid-cooled IT hardware positioned in partitions402a-402band404a-404b, and the heat exchanger within the door rotates together with the door. In one embodiment cooling doors410aand410bhave the construction shown inFIG.2C.

In the illustrated embodiment a fan unit414is positioned in a fan region between enclosures402and404, although in other embodiments there can also be fan units in between partitions within an enclosure, as shown inFIGS.3A-3B. Other embodiments need not include the fan unit, and embodiments without fan unit414need not include a fan region to house the fan unit. In embodiments with more than two enclosures, not every pair of enclosures need have a fan unit between them. In the illustrated embodiment fan unit414pulls in air or entrains from the sides of enclosures402and404and exhausts it out the rear side of the rack. As with IT racks200and300, in some embodiment of rack400fan unit414can be omitted, and in embodiments that omit the fan unit, the fan region between enclosures402and404can also be omitted.

FIGS.5A-5Billustrate another embodiment of an IT rack500. IT rack500is in most respects similar to IT rack200(seeFIGS.2A-2B): it has a front and a rear and includes an equipment enclosure202divided into vertically-oriented partitions204aand204b, although other embodiments can have more or less partitions than shown. Each partition204is an IT region adapted to receive one or more pieces of liquid-cooled IT hardware, such as servers, routers, and the like, as shown for instance inFIG.1. Each piece of IT hardware can, for example, include a chassis206within which are housed electronics and liquid cooling components208. Rack500can also include an optional fan unit214in a fan region between partitions204aand204b.

The primary difference between rack500and racks200,300, and400is the motion of the cooling door. Rack500includes a cooling door502positioned at the back of the enclosure, and the cooling door can move between a first (closed) position across the rear of the enclosure (FIG.5A) to a second (open) position where it projects from the side of the enclosure (FIG.5B). Cooling door502includes one or more rails504that slide through a guide506, so that instead of rotating between its first and second positions about a hinge, cooling door502slides between the first and second positions through guide506along rails504. In one embodiment cooling door502can have a construction substantially as shown inFIG.2C, with the hinges replaced by rails at the top and/or bottom of the door. Connections between liquid-cooled IT equipment in the partitions can be made using flexible hoses in on example. Because there are fluid connections between the cooling door and the liquid cooling components inside the enclosure, either through a distribution manifold or not, corresponding flexible hoses should be sufficient long enough to support moving cooling door502from one position to another. Other type of design can also be used in other embodiments. Other embodiments of rack500can include multiple cooling doors, analogously to racks200,300, or400, and rack500has all the same operating modes as racks200,300, and400.

FIG.6illustrates an embodiment of a data center600. Data center600includes a data center facility602. Although not shown in the drawing, data center600can include an air cooling system coupled to data center facility602to control the air flow and air temperature within the facility. A plurality of IT racks is positioned within data center facility602, arranged in at least one row. The illustrated embodiment of data center600has two rows, each with two racks, but other embodiments can have a different number of rows than shown and each row can have a different number of racks than shown. In the illustrated embodiment, the racks are all IT racks400(seeFIG.4), but other embodiments of data center600can use other rack embodiments such as racks200,300, or500. In still other embodiments, all racks in the row need not be the same type of rack. The airflow through each rack during operation is illustrated by the dashed arrows on the two leftmost racks in the figure.

In data center600, the racks within each row are spaced apart from each other. The spacing between racks is adjusted to accommodate the combined widths of cooling doors on neighboring racks—i.e., the widths of cooling doors on each rack closest to the cooling door of the neighboring rack. In the top row in the figure, for instance, each rack400_1and400_2has a pair of cooling doors410aand410b. Racks400_1and400_2are spaced apart by a distance W, which is substantially the sum of the distance w1by which cooling door410bof rack400_1projects from the side of rack400_1and the distance w2by which cooling door410aof rack400_2projects from the side of rack400_2. In other words, W≅w1+w2. In the illustrated embodiment, width W is selected so that the side of each cooling door abuts or nearly abuts the side of the neighboring cooling door—e.g., cooling door410bof rack400_1abuts cooling door410aof rack400_2—to create containment region604. But in other embodiments the spacing between racks can be different and the cooling doors need not abut. In the illustrated embodiment every rack has both its cooling doors in their second position with β=180 degrees. By spacing the IT racks in each row as described above, the equipment enclosures400and their cooling doors410form a containment region604between rows. Distance H between rows of racks can be selected to be large enough to accommodate the rotation of the cooling doors about their hinges.

In operation, cool air from the data center facility602flows through each enclosure and partition, and through each cooling door, removing heat from the electronics within each enclosure, as illustrated by the dashed arrows in the figure. Hot air exiting from the interiors of the equipment enclosures and from the heat exchangers in each cooling door flows into containment region604, from which it can then be directed to the facility's climate control equipment to be exhausted from the facility or cooled and reintroduced into the facility as new cooling air. Containment region604, then, functions as a hot aisle in a hot aisle/cold aisle arrangement which is used to contain the airflow and separate the airflow for the room for two regions. The cooling airflow is separated and used for cooling the racks and the cooling doors separately and simultaneously, so that the cooling doors function as heat expansion units that use the fluid circulating within the closed loop to transferring some portion of the heat within the IT and rack to the cooling door. Therefore, the heat transfer area between the heat load and the cooling air is increased significantly and the facility airflow can be used more efficiently. This can also be understood as increasing the rack density in a single rack, however, while using the cooling door to separate the heat or expand the heat exchange area for better thermal management.

FIG.7. illustrates an embodiment of a data center700. Data center700is in most respects similar to data center600: it includes a data center facility702and, although not shown in the drawing, it can include an air cooling system coupled to data center facility702to control the air flow and air temperature within the facility. Data center700can also include a liquid cooling system to circulate cooling fluid through the individual racks and their cooling doors. A plurality of IT racks is positioned within data center facility702, arranged in at least one row. The illustrated embodiment of data center700has two rows, each with three racks, but other embodiments can have a different number of rows than shown and each row can have a different number of racks than shown. In the illustrated embodiment, the racks are all IT racks200(seeFIG.2), but other embodiments of data center700can use other IT rack embodiments such as racks300,400, or500. In still other embodiments, all racks in the row need not be the same type of rack. The airflow through each rack during operation is illustrated by the dashed arrows on rack200_1in the figure.

In data center700, the racks within each row are spaced apart from each other. The spacing between racks is adjusted to accommodate the widths of the cooling doors on each rack. In the top row in the figure, for instance, racks200_1through200_3each have a cooling door210. As illustrated by racks200_2and200_3, racks in each row are spaced apart by a distance W, which is substantially the width w1by which cooling door210of rack200_3projects from the side of rack200_3. In other words, W≅w1. With this spacing, each cooling door abuts or nearly abuts the neighboring rack. Distance H between rows of racks can be selected to be large enough to accommodate the rotation of cooling doors210about their hinges. In the illustrated embodiment every rack has both its cooling doors210in their second position with β=180 degrees. By spacing the IT racks in each row as described above, the equipment enclosures200and their cooling doors210form a containment region704between rows, similar to data center600.

Operation of data center700is similar to operation of data center600. Cool air from the data center facility702flows through each enclosure and partition and through each cooling door, removing heat from the electronics within each enclosure, as illustrated by the dashed arrows in the figure. Hot air exiting from the interiors of the equipment enclosures and from the heat exchangers in each cooling door flows into containment region704, from which it can then be directed to the facility's climate control equipment for cooling and possible reintroduction into the facility as new cooling air. Containment region704functions as a hot aisle in a hot aisle/cold aisle arrangement. Both FIG.6andFIG.7show hot aisle containment, but different containment solutions may can be coupled in actual use.

FIG.8. illustrates an embodiment of a data center800. Data center800is in most respects similar to data centers600and700: it includes a data center facility802and, although not shown in the drawing, can include an air cooling system coupled to data center facility802to control the air flow and air temperature within the facility. Data center800can also include a liquid cooling system to circulate cooling fluid through the individual racks and their cooling doors. A plurality of IT racks is positioned within data center facility802, arranged in at least one row. The illustrated embodiment of data center800has two rows, each with two racks, but other embodiments can have a different number of rows than shown and each row can have a different number of racks than shown. In the illustrated embodiment, the racks are all IT racks400(seeFIG.4), but other embodiments of data center700can use other rack embodiments such as racks200,300, or500. In still other embodiments, all racks in the row need not be the same type of rack.

Data center800illustrates the deployment flexibility of the IT racks described above. Each racks can be operated with a different cooling door configuration than previously described, and all racks need not operate with the same cooling door configuration. In the top row in the figure, for instance, each rack400_1and400_2has a pair of cooling doors410aand410b. Racks400_1and400_2are spaced apart by a distance W, as described above forFIG.6, to allow for movement of the cooling doors410to their second (open) position. But despite being placed relative to each other to accommodate the second (open) positions of their cooling doors with β=180 degrees, rack400_1and400_2both keeps their cooling doors in their first (closed) position with β=0 degrees. Racks in a row can also be operated with some cooling doors in their closed position and some in their open position. For instance, as shown in the bottom row in the figure, racks400_3and400_5each have one door in the closed position and one door in the open position with β=80 degrees, while rack400_4keeps both its cooling doors in their closed position with β=0 degrees. Rack400_4also omits the fan unit, illustrating that it can be used or omitted as necessary to tailor the heat transfer from each rack or to tailor the spacing between racks. The fan unit used in400_3is dedicated for the rack on its left, while the fan unit in400_5is dedicated for the rack on its right.

FIG.9illustrates an embodiment of a data center900. Data center900is in most respects similar to data centers600,700, and800: it includes a data center facility902and, although not shown in the drawing, can include an air cooling system coupled to data center facility902to control the air flow and air temperature within the facility. Data center900can also include a liquid cooling system to circulate cooling fluid through the individual racks and their cooling doors. A plurality of IT racks is positioned within data center facility902, arranged in at least one row. The illustrated embodiment of data center900has two rows, each with four racks, but other embodiments can have a different number of rows than shown and each row can have a different number of racks than shown. In the illustrated embodiment, the racks are all IT racks300(seeFIGS.3A-3B), but other embodiments of data center900can use other rack embodiments such as racks200,300, or500. In still other embodiments, all racks in the row need not be the same type of rack. In the figure, cooling doors are illustrated in their closed positions (solid outline) and open positions (dashed outline), but in any given configuration each cooling door will either be open or closed.

Data center900again illustrates the deployment flexibility of the IT racks described above. When the dimension H of containment area/hot aisle904is a limitation, a rack such as rack300with two cooling doors can be a better option. Using two cooling doors may provide more cooling flexibilities on the rack power density non-uniform management. The cooling doors are presented in solid lines in their closed position (β=0), and in dashes lines in their open position (β=180 in this embodiment). The spacing between rack is as describe above for data center600, so that when all cooling doors are in their open position they abut or nearly abut so that the cooling doors and enclosures create a containment area/hot aisle containment904.

In a data center that is purely air cooled, high-density racks are hybrid cooled using air and liquid: liquid carrying heat load is recirculated to the cooling door, and the door is cooled using facility airflow. If the rack itself is only air cooled, then there are two possibilities. First, the cooling door can be used closed but fluidly coupled to an external data center cooling fluid loop if available, in which case the cooling door functions as a rear door heat exchanger. This is being operated as hybrid cooled system with the IT rack being air cooled. Second, the cooling door can be used open but with a blanking panel, in which case facility airflow is used only for cooling the racks directly.

As previously mentioned, the lifetime of the data center and IT equipment are different. For example, consider an air-cooled data center built for 160 kW. If the power rating for each first generation IT and rack configuration is 10 kW, then the two rows shown inFIG.9can be populated with 16 racks perfectly. But if in the second generation rack power goes up to 20 kW, embodiment of the described IT rack can deploy eight 20 kW racks and at the same time use the cooling system in a most efficient manner. Therefore, the described rack embodiments provide a highly flexible platform for server deployments in data centers.

Other embodiments of the described IT racks and data centers are possible besides the ones described above. For instance:The internal system design, such as the heat exchanger, pumping system and the individual liquid cooling loop for each node may be different.The solution can be used in either an air cooling data center or liquid cooling data centers; in a liquid cooling data center, cooling fluid is supplied to the rack or the cooling door; therefore, the deployment method and rack internal loop design may be different.Additional fan systems or structural may be used for airflow management and better efficiency.

The above description of embodiments is not intended to be exhaustive or to limit the invention to the described forms. Specific embodiments of, and examples for, the invention are described herein for illustrative purposes, but various modifications are possible.