Rack information handling system having modular liquid distribution (MLD) conduits

A direct-interface liquid-cooled (DL) Rack Information Handling System (RIHS) includes liquid cooled (LC) nodes each comprising a chassis received in a respective chassis-receiving bay of a rack and containing heat-generating functional components. Each LC node is configured with a system of conduits to receive direct injection of cooling liquid to regulate the ambient temperature of the node and provide cooling to the functional components inside the node by removing heat generated by the heat-generating functional components. A cooling subsystem has a liquid rail formed by more than one node-to-node, Modular Liquid Distribution (MLD) conduits, each including first and second terminal connections attached on opposite ends of a central conduit that can be rack-unit dimensioned. The MLD conduits seal to and enable fluid transfer between a port of a selected LC node and a port of an adjacent LC node.

BACKGROUND

1. Technical Field

The present disclosure generally relates to information handling systems (IHS), and more particular to a rack-configured IHS, having a liquid cooling subsystem and liquid-cooled nodes. Still more particularly, the disclosure is related to a liquid cooling system having a modular rack liquid distribution network of conduits.

2. Description of the Related Art

For implementations requiring a large amount of processing capability, a rack-configured (or rack) IHS (RIHS) can be provided. The RIHS includes a physical rack, within which is inserted a plurality of functional nodes, such as server (or processing) nodes/modules, storage nodes, and power supply nodes. These nodes, and particularly the server nodes, typically include processors and other functional components that dissipate heat when operating and/or when connected to a power supply. Efficient removal of the heat being generated by these components is required to maintain the operational integrity of the RIHS. Traditional heat removal systems include use of air movers, such as fans, to convectionally transfer the heat from inside of the RIHS to outside the RIHS. More recently, some RIHS have been designed to enable submersion of the server modules and/or the heat generating components in a tank of cooling liquid to effect cooling via absorption of the heat by the surrounding immersion liquid.

The amount of processing capacity and storage capacity per node and/or per rack continues to increase, providing greater heat dissipation per node and requiring more directed cooling solutions. Thus, there is a continuing need for further innovations to provide directed cooling for the individual heat generating components, both at the individual node level, as well as at the larger rack level. When designing the cooling subsystem, consideration must also be given to the different form factors of IT nodes and rack heights of the RIHS, and the ability to effectively control cooling discretely (at device or node level) and generally across the overall RIHS.

As liquid cooling improves in efficiencies and performance, data center solutions continue to focus on implementing liquid cooling at the rack level. Recently, localized liquid solutions (CPU/GPU cold plates) have been successful in removing most of the heat from these components within a server and into the facility cooling loop through direct fluid-to-fluid heat exchangers (server cooling loop to facility cooling loop) within the rack, but this method does not provide cooling to auxiliary components (such as storage devices (HDDs, memory), or critical secondary ICT equipment, such as top of the rack switch, network switches, battery backup units, or Power Supply Units (PSUs).

BRIEF SUMMARY

The illustrative embodiments of the present disclosure provides a Direct-Interface Liquid-Cooled (DL) Rack Information Handling System (RIHS), a direct-interface liquid cooling subsystem, and a method for modularly providing liquid cooling to information technology (IT) nodes within a RIHS. The IT nodes are liquid cooled (LC) nodes that contain heat-generating functional components, and the nodes can have different form factors and different cooling requirements. In one or more embodiments, the RIHS includes a rack having chassis-receiving bays. Each chassis is received into a respective chassis-receiving bay of the rack. The chassis can be a node enclosure or can be a block chassis that receives more than one node enclosure. Each LC node is configured with a system of conduits to receive direct injection of cooling liquid to regulate the ambient temperature of the node and provide cooling to the components inside the node by absorbing and removing heat generated by the heat-generating functional components. The cooling subsystem has a liquid rail formed by more than one node-to-node, Modular Liquid Distribution (MLD) conduits connecting between LC nodes provisioned in the rack. The MLD conduits comprise first and second terminal connections attached on opposite ends of a central conduit that is rack-unit dimensioned to seal to and enable fluid transfer between a port of a selected LC node and a port of an adjacent LC node.

According to at least one aspect of the present disclosure, a liquid cooling subsystem of a RIHS includes a liquid rail formed by more than one node-to-node, MLD conduits connecting between more than one LC node provisioned in a rack. Each LC node includes a chassis received in a respective chassis-receiving bay of the rack and each LC node contains one or more heat-generating functional components. Each LC node is configured with a system of conduits to receive direct injection of cooling liquid to regulate the ambient temperature of the node and provide cooling to the components inside the node by absorbing and removing heat generated by the heat-generating functional components.

According to at least one aspect of the present disclosure, a method is provided of assembling a DL RIHS that includes mounting a plurality of block liquid manifolds at a back of the rack, where each block liquid manifold has at least one rail supply port and a rail return port on an outside facing side of the block liquid manifold that respectively communicate with at least one block supply port and a block return port on an inside facing side of the block liquid manifold. The method includes inserting more than one LC nodes in receiving bays of the rack corresponding to locations of the mounted block liquid manifolds, with the supply ports and the return ports of the LC nodes and an inside facing portion of the corresponding block liquid manifold linearly aligned. The method includes sealing, for fluid transfer, the ports of a selected one or more LC nodes to the lineally aligned block supply and return ports of the inside facing portion of the block liquid manifold. The method includes attaching node-to-node interconnecting Modular Liquid Distribution (MLD) conduits between a selected pair of adjacent rail supply ports and adjacent rail return ports of the exterior face of the block liquid manifolds to form a supply conduit and a return conduit, respectively, of a scalable liquid rail. The method further includes attaching bypass conduits across the nodes to allow for continued flow of cooling liquid to the remaining nodes in the RIHS following removal of one or more of the MLD conduits serving specific nodes. The assembly enables liquid cooling to the functional components inside the LC nodes. Liquid cooling is provided to the functional components inside the LC nodes via liquid absorption of heat generated by the heat-generating functional components, followed by liquid transfer of the absorbed heat away from the inside of the node.

The above presents a general summary of several aspects of the disclosure in order to provide a basic understanding of at least some aspects of the disclosure. The above summary contains simplifications, generalizations and omissions of detail and is not intended as a comprehensive description of the claimed subject matter but, rather, is intended to provide a brief overview of some of the functionality associated therewith. The summary is not intended to delineate the scope of the claims, and the summary merely presents some concepts of the disclosure in a general form as a prelude to the more detailed description that follows. Other systems, methods, functionality, features and advantages of the claimed subject matter will be or will become apparent to one with skill in the art upon examination of the following figures and detailed written description.

DETAILED DESCRIPTION

The present disclosure generally provides a Direct-Interface Liquid-Cooled (DL) Rack Information Handling System (RIHS) providing liquid cooled (LC) information technology (IT) nodes containing heat-generating functional components and which are cooled at least in part by a liquid cooling subsystem. The RIHS includes a rack configured with chassis-receiving bays in which is received a respective chassis of one of the LC nodes. Each LC node is configured with a system of conduits to receive direct injection of cooling liquid to regulate the ambient temperature of the node. Additionally, each LC node, configured with a system of conduits, provides cooling to the components inside the node by conductively absorbing, via the cooling liquid, heat generated by the heat-generating functional components. The absorbed heat is removed (or transferred away) from within the node to outside of the node and/or the RIHS. The cooling subsystem has a liquid rail formed by more than one node-to-node, Modular Liquid Distribution (MLD) conduit, which include first and second terminal connections attached on opposite ends of a central conduit. The MLD conduits are rack-unit dimensioned to seal to and enable fluid transfer between a port of a selected LC node and a port of an adjacent LC node.

As utilized herein, the term “rack-configured” (as in RIHS) generally refers to the configuration of a large scale sever system within a physical rack having multiple chassis receiving rails for receiving specific sizes of information technology (IT) nodes, such as server modules, storage modules, and power modules. The term node generally refers to each separate unit inserted into a 1U or other height rack space within the rack. In one embodiment, operational characteristics of the various IT nodes can be collectively controlled by a single rack-level controller. However, in the illustrated embodiments, multiple nodes can be arranged into blocks, with each block having a separate block-level controller that is communicatively connected to the rack-level controller.

As illustrated by the figures and described herein, multiple processing servers or server IHSs (referred to herein as server nodes) can be included within the single RIHS. Certain aspect of the disclosure then relate to the specific LC (sever or other) nodes and the functionality associated with these individual nodes or block-level groupings of nodes, while other aspects more generally relate to the overall DL RIHS containing all of the LC nodes.

As one design detail/aspect for the present innovation, consideration is given to the fact that extreme variations can exist in server/power/network topology configurations within an IT rack. In addition to dimension variations, the thermal requirements for heat-generating functional components for power, control, storage and server nodes can be very different between types or vary according to usage. These variations drive corresponding extreme diversity in port placement, fitting size requirements, mounting locations, and manifold capacity for a liquid cooling subsystem. Further, a chassis of each node is typically densely provisioned. Lack of space thus exists to mount a discrete water distribution manifold in high-power IT racks. The present disclosure addresses and overcomes the challenges with distributing liquid cooling fluids throughout an IT rack having nodes with a large number of variations in distribution components.

The disclosure also includes the additional consideration that in addition to cooling the primary heat generating components of the rack, such as the processor, what is needed is a way to allow for cooling of secondary equipment within the rack, as well as auxiliary components that would further support utilizing the advantages of a fluid-to-fluid heat exchanger methodology. Additionally, the present disclosure provides a modular approach to utilizing an air-to-liquid heat exchanger with quick connection and scalability to allow the solution to be scalable in both 1U and 2U increments.

FIG. 1illustrates a side perspective view of an internal layout/configuration of an example Direct-Interface Liquid-Cooled (DL) RIHS100configured with a plurality of LC nodes102, according to one or more embodiments. For simplicity, the example DL RIHS presented in the various illustrations can be described herein as simply RIHS100; however, references to RIHS100are understood to refer to a DL RIHS, with the associated liquid cooling infrastructure and/or subsystems and supported LC nodes102. RIHS100includes rack104, which comprises a rack frame and side panels, creating a front-to-back cabinet within which a plurality of chassis receiving bays are vertically arranged and in which a chassis of a respective IT node102can be inserted. Rack104includes certain physical support structures (not specifically shown) that support IT gear insertion at each node location. Additional description of the structural make-up of an example rack is provided in the description ofFIGS. 2-4, which follows.

FIG. 1depicts an illustrative example of LC nodes102a-102j(collectively refer to as nodes102), with each nodes102a-102iincluding heat-generating functional components106. Additionally, RIHS100also includes an infrastructure node102jand liquid filtration node102k, which do not necessarily include heat-generating functional components106that require liquid cooling, as the other LC nodes102a-102i. In the illustrative embodiments, nodes102a-102b, and102e-102hinclude other components108that are not necessarily heat generating, but which are exposed to the same ambient heat conditions as the heat generating components by virtue of their location within the node. In one embodiment, these other components108can be sufficiently cooled by the direct injection of cooling liquid applied to the node and/or using forced or convective air movement, as described later herein. Each node102is supported and protected by a respective node enclosure107. Nodes102a-102dare further received in node receiving bays109of a first block chassis110aof a first block112a. Nodes102e-102iare received in a second block chassis110of a second block112b. In the illustrative embodiments, the nodes102are vertically arranged. In one or more alternate embodiments, at least portions of the nodes102(and potentially all of the nodes) may also be arranged horizontally while benefiting from aspects of the present innovation.

The present innovation is not limited to any specific number or configuration of nodes102or blocks112in a rack104. According to one aspect, nodes102can be of different physical heights of form factors (e.g., 1U, 1.5U, 2U), and the described features can also be applied to nodes102having different widths and depths (into the rack), with some extensions made and/or lateral modifications to the placement of cooling subsystem conduits, as needed to accommodate the different physical dimensions. As a specific example, node102iis depicted as having a larger node enclosure107′ (with corresponding different dimensions of heat-generating functional components106′) of a different number of rack units in physical height (e.g., 2U) that differs from the heights (e.g., 1U) of the other nodes102a-102hand102j-102k. RIHS100can include blocks112or nodes102selectably of a range of discrete rack units. Also, different types of IT components can be provided within each node102, with each node possibly performing different functions within RIHS100. Thus, for example, a given node102may include one of a server module, a power module, a control module, or a storage module. In a simplest configuration, the nodes102can be individual nodes operating independent of each other, with the RIHS100including at least one rack-level controller (RC)116for controlling operational conditions within the RIHS100, such as temperature, power consumption, communication, and the like. Each node102is then equipped with a node-level controller (NC)118that communicates with the rack-level controller116to provide localized control of the operational conditions of the node102. In the more standard configuration of a DL RIHS100, and in line with the described embodiments, RIHS100also includes block-level controllers (BCs)114, communicatively coupled to the rack-level controller116and performing block-level control functions for the LC nodes within the specific block. In this configuration, the nodes102are arranged into blocks112, with each block112having one or more nodes102and a corresponding block-level controller114. Note the blocks do not necessarily include the same number of nodes, and a block can include a single node, in some implementations.

A Direct-Interface Liquid Cooling (DL) subsystem (generally shown as being within the RIHS and labelled herein as120) provides liquid cooling to heat-generating functional components106via a liquid rail124under the control of the rack-level controller116, block-level controllers114, and/or node-level controllers118, in some embodiments. Rack-level controller116controls a supply valve126, such as a solenoid valve, to allow cooling liquid, such as water, to be received from a facility supply128. The cooling liquid is received from facility supply128and is passed through liquid filtration node102lbefore being passed through supply conduit130of liquid rail124. Each block112a,112breceives a dynamically controlled amount of the cooling liquid via block-level dynamic control valve132, such as a proportional valve. Return flow from each block112a,112bcan be protected from backflow by a block check valve133. The individual needs of the respective nodes102a-102dof block112acan be dynamically provided by respective node-level dynamic control valves134, controlled by the block-level controller114, which control can, in some embodiments, be facilitated by the node-level controllers118. In addition to allocating cooling liquid in accordance with cooling requirements (which can be optimized for considerations such as performance and economy), each of the supply valve126and/or dynamic control valves132,134can be individually closed to mitigate a leak. A check valve136is provided between each node102a-102jand a return conduit138of the liquid rail124to prevent a backflow into the nodes102a-102j. The return conduit138returns the cooling liquid to a facility return140.

To support the temperature control aspects of the overall system, RIHS100includes temperature sensors101that are each located within or proximate to each node102a-102j, with each gauge101connected to the node-level controller118and/or the corresponding block-level controller114. Temperature sensors101operate in a feedback control loop of the liquid cooling system122to control the amount of liquid flow required to cool the nodes102a-102j. In one or more embodiments, the rack-level controller116can coordinate performance constraints to block-level controllers114and/or node-level controllers118that limit an amount of heat generated by the heat-generating functional components106to match a heat capacity of the flow of cooling liquid in DL subsystem122. Alternatively or in addition, the rack-level controller116can coordinate cooling levels to block-level controllers114and/or node-level controllers118that in turn control the dynamic control valves132,134for absorption and transfer of the heat generated by the heat-generating functional components106by the DL subsystem122. In one or more embodiments, support controllers such as a Rack Liquid Infrastructure Controller (RLIC)142can perform management and operational testing of DL subsystem122. RLIC142can monitor pressure sensors144and liquid sensors146to detect a leak, to validate operation of a namic control valves132,134or shut-off valves such as supply valve126. RLIC142can perform close-loop control of specific flow rates within the RIHS100.

FIG. 2illustrates example LC node200of example DL RIHS100ofFIG. 1having a node enclosure208insertable into a block chassis210. For purposes of description, node200is a server IHS that includes processing components or central processing units (CPUs), storage devices, and other components. LC node200includes cooling subsystem (generally shown and represented as220) that includes a liquid-to-liquid manifold242to cool heat-generating functional components206by heat transfer from liquid provided by node-level supply conduit244, and return conduit246, according to one or more embodiments. Node-level supply conduit244and return conduit246are appropriately sized and architecturally placed relative to the other components and the dimensionality (i.e., width, height, and depth/length) of LC node200to permit sufficient cooling liquid to pass through the interior of LC the node200to remove the required amount of heat from LC node200in order to provide appropriate operating conditions (in terms of temperature) for the functional components located within LC node200. Liquid-to-liquid manifold242can include CPU cold plates248and voltage regulator cold plates250. A sled assembly grab handle252can be attached between CPU cold plates248for lifting LC node200out of block chassis210. A return-side check valve254of the return conduit246can prevent facility water from back-feeding into LC node200such as during a leak event. Flex hose links256in each of node-level supply conduit244and return conduits246can reduce insertion force for sleds into block chassis210. Sled emergency shutoff device234interposed in the supply conduit244can be a solenoid valve that closes in response to input from a hardware circuit during a sled-level leak detection event. Node-level carrier258received in node enclosure208can incorporate liquid containment structure260to protect storage device262. In the illustrative example illustrated byFIG. 2, LC node200is oriented horizontally and is viewed from above. In one or more embodiments node-level carrier258is configured to route leaked cooling liquid away from storage device262when oriented vertically.

FIGS. 3-7illustrate different exterior and rear views of an example assembled DL RIHS300. DL RIHS300includes rack304, which is a physical support structure having an exterior frame and attached side panels to create cabinet enclosure364providing interior chassis receiving bays (not shown) within which a plurality of individual node chasses (or sleds)208of functional IT nodes, such as LC node200ofFIG. 2, are received. In the description of the figures, similar features introduced in an earlier figure are not necessarily described again in the description of the later figures.

FIGS. 3-5specifically illustrate exterior views of rack304of example DL RIHS100. As illustrated, rack304includes opposing side panels366, attached to a top panel368(and bottom panel—not shown) to create the main cabinet enclosure364that includes multiple chassis receiving bays for housing LC nodes102/200. The created cabinet enclosure364includes a front access side (not shown) and a rear side. The front access side provides access to the chassis receiving bays created within the main cabinet enclosure364for receiving LC nodes102(ofFIG. 1) into rack304. Attached to the rear ends of the main opposing side panels366are opposing side panel extensions372. A louvered rear door374is hinged (or otherwise attached) to one of the side panel extensions372and includes a latching mechanism for holding the door374in a closed position, where in a closed position is relative to the otherwise open space extending laterally between opposing side panel extensions372. Side panel extensions372and louvered rear door374provide an extension to main cabinet enclosure364for housing, covering/protecting, and providing access to the modular, scalable liquid rail324of a liquid cooling subsystem322that provides liquid cooling to each LC node102(ofFIG. 1) inserted into the chassis of the main cabinet enclosure364.

FIG. 4illustrates an embodiment in which rear pipe covers476can protect portions of liquid rail324(ofFIG. 3), and specifically Modular Liquid Distribution (MLD) conduits478, from inadvertent damage as well as containing any leaks from being directed at sensitive functional components106(ofFIG. 1).

Illustrated inFIG. 5are rear pipe covers476(ofFIG. 4) of MLD conduits478(ofFIG. 4) of liquid rail324(ofFIG. 3) having different sizes. According to one aspect, the MLD conduits478(ofFIG. 4) are rack unit dimensioned pipes that form a node-to-node scalable rack liquid manifold (“liquid rail”) to distribute cooling liquid, as required, for each node102(ofFIG. 1) and through the vertical arrangement of nodes102(ofFIG. 1) within RIHS100(ofFIG. 1). In an exemplary embodiment, the cooling liquid is received from a facility supply128(ofFIG. 1) via below rack (e.g. ground level or below floor) connections580.

FIG. 6illustrates an example RIHS100, as depicted inFIG. 1, with MLD conduits478(ofFIG. 4), that are uncovered, displaying liquid rail324(ofFIG. 3). In the embodiment ofFIG. 6, cooling liquid is received from a facility supply128(FIG. 1) by below rack (e.g. ground level or below floor) connections680.

FIG. 7illustrates a second example RIHS700, wherein cooling liquid is received from facility supply128(FIG. 1) provided by an above-rack (and possibly in ceiling) connections780. Also shown byFIG. 7are air movers depicted as fan modules782adjacent to the liquid rail. These fan modules782are mounted at the back of RIHS700to draw air flow through LC nodes102providing additional cooling of LC nodes102, ofFIG. 1, (e.g., convection cooling for node components106, ofFIG. 1) that may or may not also receive direct-interface of cooling liquid, in different embodiments.

FIG. 8illustrates a more detailed view of the interconnections of the liquid cooling subsystem, at a node level and rack level within an example DL RIHS800. As shown, RIHS800is configured with LC nodes802a-802earranged in blocks (e.g., block1comprising802a-802c) and which are cooled in part by a liquid cooling system having a liquid rail comprised of MLD conduits, and in part by a subsystem of air-liquid heat exchangers, can be configured with heat-generating functional components806and that are cooled at least in part by a system of MLD conduits878a-878b, according to one or more embodiments. Illustrated within nodes802are heat-generating functional components806, such as processors, voltage regulators, etc., which emit heat during operation and or when power is applied to the component, such that the ambient temperature increases around the component, and within the node, and eventually within the block, and ultimately DL RIHS800, during standard operation. To mitigate heat dissipation (and effects thereof), and to maintain the RIHS, block, node, and functional components within proper operating temperatures, DL RIHS800is configured with a DL subsystem822. DL subsystem822includes a rack level network of liquid propagating pipes, or conduits that are in fluid communication with individual node level networks of liquid propagating conduits. Additionally, DL subsystem822collectively facilitates heat absorption and removal at the component level, the node level, the block level, and/or the rack level. The rack-level network of conduits includes a modular arrangement of a liquid rail824formed by more than one node-to-node MLD conduit878a-878bspanning (or extending) between LC nodes802provisioned in rack804.

At the top position of RIHS800, a block chassis810is received in a block chassis receiving bay870aof rack804. Within block chassis810, a first node802areceived in a first node receiving bay809aof the rack804has a vertical height of one rack unit (1U). A rack unit, U or RU as a unit of measure, describes the height of electronic equipment designed to mount in a 19-inch rack or a 13-inch rack. The 19 inches (482.60 mm) or 13 inches (584.20 mm) dimension reflects the horizontal lateral width of the equipment mounting-frame in the rack including the frame; the width of the equipment that can be mounted inside the rack is less. According to current convention, one rack unit is 1.75 inches (44.45 mm) high. A second node802breceived in a second node receiving bay809bof the rack104(ofFIG. 1) has a vertical height of 1U. A third node802creceived in a third node receiving bay809cof the rack804has a vertical height of 1U. A fourth node802d, infrastructure node802b, is received in a second block chassis receiving bay870bof rack804and has a vertical height of 1U. Infrastructure node802bcan contain functional components such as a rack-level controller816. A fifth node802eis received in a third chassis receiving bay870cand has a vertical height of 2U. A sixth node802f, which provides a Rack Filtration Unit (RFU)871, is received in a fourth block chassis receiving bay870dof the rack804. Infrastructure node802and RFU871are examples of nodes802that may not require liquid cooling. A cascading liquid containment unit890is received in a fifth chassis receiving bay870eand includes liquid sensor897.

MLD conduits878aof 1U can be used to connect nodes of 1U vertical spacing. Because of the additional 1U separation of LC nodes802cand802eby inclusion of infrastructure node802d, MLD conduit878bbetween the third and fifth nodes802c-802dis dimension 2U to accommodate the increased spacing. MLD conduits878a-878bcan thus support different heights (1U to NU) of IT components.

Each MLD conduit878a-878bincludes first and second terminal connections883,884attached on opposite ends of central conduit885that is rack-unit dimensioned to seal to a port of LC node802and enable fluid transfer between a port of a selected LC node802and a port of an adjacent LC node802. InFIG. 8, facility supply828and facility return840are respectively located at the intake end of liquid rail824and the exhaust end of liquid rail824. The actual location of facility supply828and facility return840can be reversed. Alternatively, facility supply828and facility return840can be located above the RIHS800or both conduits can be located on opposite sides of the RIHS800in alternate embodiments.

Liquid cooling subsystem822includes a liquid infrastructure manager controller (LIMC)886which is communicatively coupled to block liquid controllers (BLCs)887to collectively control the amount of cooling liquid that flows through the RIHS800and ultimately through each of the nodes802in order to effect a desired amount of liquid cooling at the component level, node level, block level, and rack level. For clarity, LIMC886and BLCs887are depicted as separate components. In one or more embodiments, the liquid control features of the LIMC886and BLCs887can be incorporated into one or more of the rack-level controller816, block-level controllers820, and the node-level controllers818. As illustrated inFIG. 1and previously described, each of the LIMC886and BLCs887are connected to and respectively control the opening and closing of flow control valves that determine the amount of flow rate applied to each block and to each node within the specific block. During cooling operations, one of LIMC886and BLC887causes a specific amount of liquid to be directly injected into the intake conduits of the LC node802, which forces the cooling liquid through the system of conduits within the LC node802to the relevant areas and/or functional components/devices inside the nodes802to absorb and remove heat away from the inside of the node and/or from around the components within the node.

As another aspect, the present disclosure provides a modular approach to utilizing air-to-liquid heat exchanger888with quick connection and is scalable in both 1U and 2U increments. In one or more embodiments, DL cooling subsystem822can include a plurality of air-to-liquid (or liquid-to-air) heat exchangers888that facilitate the release of some of the heat absorbed by the exhaust liquid to the surrounding atmosphere around the RIHS100(ofFIG. 1). Air-to-liquid heat exchangers888can be integral to block liquid manifold889that, along with the MLD conduits878a-878b, form scalable liquid rail824. One aspect of the present disclosure is directed to providing scalable rack-mounted air-to-liquid heat exchanger888for targeted heat rejection of rack-mounted equipment to DL cooling subsystem822. Hot air899from auxiliary components, such as storage device895, would be pushed through the air-to-liquid heat exchanger888, and the resulting energy would transfer to liquid rail824and be rejected to a facility cooling loop, represented by the facility return840.

RIHS800can include variations in LC node802that still maintain uniformity in interconnections along liquid rail824formed by a chassis-to-chassis modular interconnect system of MLD conduits878a-878b. With this scalability feature accomplished using MLD conduits878a-878b, cooling subsystem822of the RIHS800allows each block chassis810to be a section of a scalable manifold, referred herein as liquid rail824, eliminating the need for a rack manifold. The scalability of liquid rail824enables flexible configurations to include various permutations of server and switch gear within the same rack (rack804). MLD conduits878a-878bcan comprise standardized hoses with sealable (water tight) end connectors. Thus, the rack liquid flow network can encompass 1 to N IT chassis without impacting rack topology, space constraints, and without requiring unique rack manifolds. Additionally, according to one aspect, the MLD conduits are arranged in a pseudo daisy chain modular configuration, which allows for unplugging of one MLD conduit from one rack level without affecting liquid flow to and cooling of other rack levels.

The system of conduits extending from node intake valve834into each LC node802enables each LC node802to engage to block liquid manifold889. Block chassis810or node enclosure808of each LC node102provides the intake and exhaust conduit connections to engage to respective terminals of MLD conduits878a-878bwithin the MLD network provided by liquid rail824. For example, where nodes802are designed as sleds, node enclosure808would be a sled tray, and each block would then include more than one sled tray received into block chassis810, forming the extensions of block liquid manifold889. Alternatively, the node enclosure808can be a single node chassis such as one of nodes802c-802f.

Supply and return bypass tubes890,891of each block liquid manifold889are connected by MLD conduits878a-878bto form supply rail conduit830and return rail conduit838. Due to constraints in the spacing within the figure, the tubing that extends from supply and return bypass tubes890,891are not shown, and the valves are shown as if connected directly to the bypass.FIG. 9provides a more accurate view of this features of the disclosure, with conduits extended into the respective supply and return valves at each block. Also, for clarity,FIG. 8illustrates the return rail conduit838separately. Liquid rail824enables multiple types of devices to be coupled together, each receiving an appropriately controlled portion of cooling liquid capacity. In one embodiment, liquid cooling subsystem822is passively pressurized by attaching MLD supply conduit892ato facility supply828and an MLD return conduit892bto facility return840. Liquid flow from supply rail conduit830to return rail conduit838of liquid rail824can be controlled based upon factors such as a temperature of the liquid coolant, detected temperature within LC nodes802, air temperature inside or outside of DL RIHS800, etc.

In an exemplary embodiment, the scalable rack manifold provided by liquid rail824is formed in part by MLD conduits878a-878bthat run vertically in the back of the RIHS800with quick disconnects on the front and rear face of block liquid manifold889that allows for IT/infrastructure equipment respectively to be plugged into both front and back sides of the block liquid manifold889. For example, LC nodes802, such as server modules, can plug into the front side and fan modules882can plug onto the back side of block liquid manifold889. This also allows for other liquid cooled devices such as LC Power Distribution Units (PDUs) to be plugged into the cooling liquid supply rail conduit830and return rail conduit838of liquid rail824. Thereby, a rack hot pluggable cooling interface is created for any rack-mounted equipment.

Cooling subsystem822can support an embedded liquid-to-liquid heat exchanger manifold842, such as in LC node802c. Node liquid-to-liquid heat exchangers are provided for rejecting heat from one fluid source to a secondary source. One aspect of the present disclosure solves the problems that many shared-infrastructure IT systems (e.g., blade chassis) do not have adequate space to accommodate a liquid-to-liquid heat exchanger. Unlike with generally-known systems that rely upon liquid heat transfer having to exchange heat with an external liquid-to-liquid heat exchanger, the present disclosure enables on-rack liquid-to-liquid heat exchanger that does not require any of the vertical chassis space. Additionally, the present disclosure provides these benefits without requiring a central distribution unit (CDU), which takes up datacenter floor space. One aspect of the present disclosure provides embedded heat exchanger manifold842having a common heat transfer plate and a shared bulk header to create a combined liquid distribution manifold that includes a secondary liquid coolant for absorbing heat through the shared bulk header. In particular, the combined embedded heat exchanger manifold842rejects heat within shared node enclosure808such as node802cto a secondary liquid coolant. Internal node supply844and return conduits846of a manifold built on top of a heat exchanger core allow heat transport within manifold842. In one embodiment, closed system pump898can use a first coolant to cool a high thermal energy generating functional component such as a CPU or voltage regulator.

Additionally, the liquid cooling subsystem822also includes a filtration system or unit871, which prevents chemical impurities and particulates from clogging or otherwise damaging the conduits as the fluid passes through the network of conduits. According to one aspect of the disclosure, liquid cooling subsystem822provides RFU871in fluid connection with the intake pipes from facility supply828. In at least one embodiment, RFU871includes a sequenced arrangement of liquid filters within a full-sized sled that can be removably inserted by an end user into one of the receiving slots of rack804. In one embodiment, the RFU871is located on an infrastructure sled having rack-level controllers and other rack-level functional components. In at least one embodiment, the entirety of the sled is filed with components associated with RFU871. Thus, it is appreciated that the RFU871may occupy the entire area of one vertical slot/position within the chassis. Alternate locations of the RFU871can also be provided, in different embodiments, with an ideal location presenting the intake port of the RFU871in close proximity to a connection to facility supply828to directly receive the facility supply828prior to the liquid being passed into the remainder of the conduits of the liquid cooling subsystem822. It is appreciated that if the system was capable of completing all heat exchange within the rack, then sealing the rack would be feasible and would reduce and/or remove any requirements for filtration and/or allocation of rack space for RFU871.

Liquid cooled compute systems use the high heat transport capacity of water. However, the disclosure recognizes and addresses the fact that with liquid introduced into an electronic enclosure, there is a potential for leaks that can cause catastrophic system failure. Also, in some instances, a leak can create an electronic short with a resulting exothermal reaction causing permanent damage to the DL RIHS800. To mitigate such risks, as one design feature, node-level carrier893can include a trench/gutter system for use as liquid containment structure894. In one embodiment, the gutter system can also incorporate an absorbent material that can accumulate sufficient amounts of liquid from small leaks to enable external sensing of the leak. Advantageously, the carrier893can also be thermally conductive to serve as a heat sink for components such as storage devices895. In one embodiment, another leak detection solution that can be incorporated into the LC node802involves use of a solenoid to create an event when additional current is applied, due to water pooling around the solenoid. Barriers on carrier893can be specifically designed to contain a liquid leak and assist in funneling the liquid through the gutter system. Liquid rail824can also be provided with leak containment and detection. In one or more embodiments, removable pipe covers876are sized to be mounted around respective MLD conduits878a-878band can include liquid sensors897for automatic alerts and shutdown measures.

In one or more embodiments, DL RIHS800further incorporates a node-level liquid containment structure890with a cascading drain runoff tubing network896to a rack-level cascading liquid containment structure894. In one or more embodiments, the DL RIHS800further incorporates leak detection response such as partial or complete automated emergency shutdown. Liquid sensors (LS)897at various cascade levels can identify affected portions of DL RIHS800. Containment and automatic shutoff can address the risks associated with a leak developing in the DL cooling system822.

FIG. 9Aillustrates a more detailed view of DL subsystem920associated with example DL RIHS900. Within DL RIHS900, each LC node902includes chassis910received in a respective chassis-receiving bay970of rack904. Each LC node902contains heat-generating functional components906. Each LC node902is configured with a system of internal supply conduit944and return conduit946, associated with embedded heat exchanger manifold942. Embedded heat exchanger manifold942receives direct injection of cooling liquid to regulate the ambient temperature of LC node902. A node-level dynamic control valve934and node-level return check valve936control an amount of normal flow and provide shutoff and/or otherwise mitigate a leak. Cooling subsystem920provides cooling to heat-generating functional components906inside the LC node902by removing heat generated by heat-generating functional components906. Liquid rail924is formed from more than one node-to-node, MLD conduit978between more than one LC node902within in rack904. MLD conduits978includes first terminal connection983and second terminal connection984. First terminal connection983and second terminal connection984are attached on opposite ends of central conduit985. Central conduit985is rack-unit dimensioned to directly mate and seal to and enable fluid transfer between a selected pair of rail supply ports917and/or rail return ports919of a selected LC node902and an adjacent LC node902.

The cooling subsystem920includes block liquid manifolds989mountable at a back side of the rack904. Each block liquid manifold has at least one rail supply port917and at least one rail return port919on an outside facing side of the block liquid manifold989. The at least one rail supply port917and the at least one rail return port919respectively communicate with at least one block supply port921and a block return port923on an inside facing side of the block liquid manifold989. LC nodes902are insertable in receiving bays970of rack904corresponding to locations of the mounted block liquid manifolds989. Block supply ports921and block return ports923of the LC nodes902and an inside facing portion of the corresponding block liquid manifold989are linearly aligned. The linear alignment enables direct sealing, for fluid transfer, of the lineally aligned inside manifold supply ports925and return ports927to the inside facing portion of the block liquid manifold989. In one or more embodiments, block supply port921sealed to the internal manifold supply port925communicates via supply bypass tube990to two rail supply ports917. Block return port923sealed to internal manifold return port927communicates via return bypass tube991of the respective block liquid manifold989to two rail return ports919. Fan modules982mounted respectively onto back of block liquid manifold989have apertures to expose rail supply917and return ports919. Additionally, fan modules982draw hot air999from LC nodes902through an air-liquid heat exchanger988in block liquid manifold989.

In one or more embodiments, supply liquid conduit992ais attached for fluid transfer between facility supply928and rail supply port917of block liquid manifold989of RIHS900. A return liquid conduit992bcan be attached for fluid transfer between rail return port919of block liquid manifold989to facility return940.FIG. 9Afurther illustrates that the fluid connection to facility supply928includes RFU971. To prevent contamination or damage to cooling subsystem920, RFU971is received in bay970of rack904and includes input port929connected via supply liquid conduit992ato facility supply928. The RFU971includes output port931that is connected to MLD conduit978of supply rail conduit930. Liquid rail924also includes return rail conduit938. RFU971controls two external emergency shutoff valves933for flow received from the input port929that is provided via hot-pluggable disconnects935to respective replaceable filtration subunits (“filters”)937. The separation of the intake fluid across dual shutoff valves933and filters937enables the supply of cooling liquid to continue even when one of the filters is removed or clogged up (preventing the passage of cooling liquid) and/or one of the shutoff valves933is closed off. The cooling liquid flows in parallel to two replaceable filtration subunits937, automatically diverting to the other when one is removed for cleaning or replacement. Thereby, filtration and cooling of RIHS900can be continuous even while servicing one of filters937. Back-flow is prevented by check valve939that allows normal flow to exit to output port931. Differential pressure sensor944measures the pressure drop across filters”)937and provides an electrical signal proportional to the differential pressure. According to one aspect, a Rack Liquid Infrastructure Controller (RLIC)942can determine that one filter937is clogged if the differential pressure received from differential pressure sensor944falls below a pre-determined value.

In one or more embodiments, RIHS900can provide hot-pluggable server-level liquid cooling, an integrated leak collection and detection trough, and an automatic emergency shut-off circuit. At a block level, RIHS900can provide embedded air-to-liquid heat exchange, and dynamic liquid flow control. At a rack level, RIHS900can provide facility-direct coolant delivery, a scalable rack fluid network, a rack filtration unit, and automated rack flow balancing, and a service mode.

According to one embodiment, liquid rail924includes a series of secondary conduits, such as supply divert conduit997and return divert conduit998that provides a by-pass fluid path for each of MLD conduits978. In operation, divert conduit997allows for the removal of corresponding MLD conduit978, thus removing the flow of cooling liquid to the particular block of nodes, without interrupting the flow of cooling liquid to the other surrounding blocks of computer gear. For example, a particular MLD conduit978can be replaced due to a leak. For another example, a block liquid manifold989can be replaced. The inclusion of divert conduits997,998thus enables rapid servicing and maintenance of block liquid manifold989and/or nodes within block chassis without having to reconfigure the MLD conduits978. In addition, the RIHS900can continue operating as cooling liquid continues to be provided to the remainder of the blocks that are plugged into the liquid rail. Re-insertion of the MLD conduit978then reconnects the flow of cooling liquid to the block for normal cooling operations, and shuts off the diverted flow of cooling liquid. In an exemplary embodiment, the MLD conduits978provide a quick disconnect feature that interrupts flow when not fully engaged to a respective port917,919,921,923. Disconnection of an MLD conduit978interrupts flow in a primary portion of the liquid rail924for either supply or return, shifting flow through one or more divert conduits997to provide cooling liquid to the other block liquid manifolds989. In one or more embodiments, a manual or active shutoff valve can interrupt flow on either or both of the primary or divert portions of the liquid rail924.

FIG. 9Billustrates the DL RIHS900having three block chasses910, each having a block liquid manifold989represented by a supply bypass tube990and a return bypass tube991. For clarity, a single LC node902is received in a respective block chassis910. A dynamic control valve, such as a proportional valve (PV)934, controls an amount of cooling liquid flow that is directed through an LC heat-exchange component942to cool a functional component (FC)906. The warmed cooling liquid flow passes through a check valve (CV)936to join a bypass flow in the respective return bypass tube991A LC subsystem920provides the cooling liquid flow. Supply and divert conduits997,998are not required to handle the flow in this nominal case with a complete liquid rail924.FIG. 9Cillustrates one LC node920removed. The empty block chassis910provides routine bypass flow through the supply and return bypass tubes990,991as part of the primary flow path. the supply and divert conduits997,998are not required to handle the flow in a secondary path.FIG. 9Dillustrates a supply MLD conduit989and a return MLD conduit989removed for replacement. The primary flow path is interrupted by removal of the MLD conduits989. The supply and return divert conduits997,998provide a secondary flow path to block chasses910that are downstream of the removed MLD conduits989. The LC node920that remains in the affected block chassis910continues to receive cooling liquid and can continue full operation.FIG. 9Eillustrates removal of a block chassis910. The supply and return divert conduits997,998provide a secondary flow path to provide cooling liquid flow to the remaining block chasses910.

FIG. 10illustrates a more detailed view of the internal makeup of the rails and other functional components of the cooling subsystem1022of example RIHS1000. According to one embodiment, cooling subsystem1022also includes air movers and/or other devices to provide for forced air cooling in addition to the direct injection liquid cooling. As shown byFIG. 10, at least one fan module1082is rear mounted to a block liquid manifold1089in which an air-to-liquid heat exchanger (or radiator)1088is incorporated. The fan module1082provides air movement through the chassis1010and/or node enclosure1008of the node1002as well as through the air-to-liquid heat exchanger1088. Each block liquid manifold1089includes a supply bypass tube1090and a return bypass tube1091through which a dynamically determined amount of cooling liquid is directed into the respective node1002while allowing a bypass flow to proceed to the next node/s1002in fluid path of the intake flow. Fan module1082includes apertures1047through which the supply and return bypass tubes1090,1091are extended, in one embodiment. Nodes1002are connected into the back side of the block liquid manifold with the ends of intake and exhaust liquid transfer conduits in sealed fluid connection with bypass tubes1090and1091respectively.

FIG. 11illustrates the example DL RIHS1000with MLD conduits878a-878b(ofFIG. 8) of two different multiples of rack units in dimension, according to one or more embodiments. Terminal connections1083,1084with connecting central conduit1085can be formed from hose materials with molded perpendicular bends.FIG. 12illustrates the example RIHS1000including bottom-feed facility supply and return MLD conduits1292a,1292b, according to one or more embodiments.FIG. 12also illustrates two service buttons1210located at the back-lower portion of the rack. Service buttons1210are located on and/or in communication with RFU102k(FIG. 1), features of which are presented in greater detail inFIGS. 13A-13B. Service buttons1210enable manual triggering of a service mode of DL RIHS, that allows for removal and re-insertion of one or more nodes and/or other components plugged into the fluid rail without experiencing a significant amount of hydraulic force and without having to shut down the entire DL RIHS to implement the service of one component.

FIGS. 13A-13Billustrate additional structural details of hot pluggable RFU1371, which includes filters for filtering out contaminants in order to protect the liquid transfer conduits from clogging and/or chemical deterioration. RFU1371includes a front air purging connection1303for temporary bleeding of pressure for filter tray installation. RFU1371includes a rear air purging connection1305for temporary bleeding of pressure for block liquid interconnect installation. RFU1371includes hot-pluggable filter drawer1311and drawer1313that are plumbed in parallel for continued operation during servicing of drawer1311or drawer1313. RFU1371includes node chassis1308insertable into a rack of an RIHS. At least one node supply port1349and at least one node return port1351are positioned on an inserted side1353of the node chassis1308to seal for fluid transfer respectively to a facility liquid supply conduit and a rail supply conduit of the liquid rail of a liquid cooling system for the RIHS. First filtration subunit1355and second filtration subunit1357are housed in hot-pluggable filter drawer1311and drawer1313, which are connected in parallel fluid communication within node chassis1308. Each filtration subunit1355and filtration subunit1357is individually disengageable from node chassis1308for maintenance or replacement, while the other filtration subunit1355or filtration subunit1357remains engaged in the node chassis and continues liquid filtration. A liquid coolant diversion network1359diverts liquid flow to the other filtration unit1355or filtration unit1357for continuous filtration of contaminants and/or particulates from the cooling liquid received from the supply side, when one filtration unit1355or filtration unit1357is removed.

FIG. 13Billustrates purge check valves1359that prevent liquid short circuit through purge solenoid valve1310of RFU1371. In an exemplary embodiment, dual node supply ports1349a,1349band dual node return port1351a,1351bsupport two independent feeds with external solenoids that are powered and/or controlled from RFU1371for filter drawers1311,1313, respectively. According to one aspect, purge solenoid valve1310is triggered by a rack liquid infrastructure controller (RLIC) or other service mode controller to open and dispense a specific amount of liquid from within the liquid cooling system of conduits to reduce the overall pressure of liquid within the system. In one embodiment, the amount of liquid released by purge solenoid valve1310can be variable based on the pressure within the system of conduits as measured by one or more liquid pressure sensors (not specifically shown). RFU1371includes a differential pressure sensor that measures a pressure drop across a filter and provides an electrical signal proportional to the differential pressure. According to one aspect, the RLIC can determine that the filter is clogged if the differential pressure received from differential pressure sensor1184C falls below a pre-determined value.

FIG. 14illustrates a method1400of assembling a DL RIHS, such as one of RIHS100,800,900, and/or1000. In one or more embodiments, method1400includes forming a block liquid manifold having a supply bypass tube and a return bypass tube that terminate respectively in a pair of supply ports and a pair of return ports to form a portion of a supply conduit and a return conduit respectively of a liquid rail (block1402). The method includes configuring a rack enclosure to receive more than one LC nodes each configured with a system of internal conduits to receive direct injection of cooling liquid from a liquid rail side of the rack chassis to regulate the ambient temperature of the node (block1404). The method1400includes mounting more than one LC node and corresponding block liquid manifolds in a rack with the respective supply ports linearly aligned and the return ports linearly aligned, wherein the supply bypass tube and the return bypass tube of a respective block liquid manifold seals via a dynamic control valve to the internal conduits of a corresponding LC node (block1406). The method1400optionally includes providing selected MLD conduits in a first number of rack units in size that can correspond to the block chassis size and a forming selected conduits in a second number of rack units in size, where each MLD conduit is formed having first and second terminal connections between a central conduit (block1408). In one embodiment, the MLD conduits are purchased in correct dimensions from a third party supplier. The method1300includes attaching node-to-node interconnecting MLD conduits between a selected pair of adjacent rail supply ports and adjacent rail return ports of the exterior face of the block liquid manifolds to form a supply conduit and a return conduit respectively of a scalable liquid rail that enables cooling to the functional components inside the LC nodes by enabling liquid transfer through each node to absorb and remove heat generated by heat-generating functional components inside of the node (block1410). The method further includes attaching bypass conduits across the chasses to allow for continued flow of cooling liquid to the remaining blocks in the RIHS following removal of one or more of the MLD conduits serving specific blocks along the rail. Removal of MLD conduits for a specific block chassis can occur during servicing of one or more nodes in that particular block, as one example. The bypass conduits also allow the facility supply and return to be able to connect in a reverse configuration, where the direction of liquid flow is reversed. The method1400includes optionally attaching a supply liquid conduit for fluid transfer to one port of one block liquid manifold of the RIHS to a facility supply and attaching a return liquid conduit for fluid transfer to one port of one block liquid manifold of the RIHS to a facility return (block1412). Then method1400ends.

FIG. 15illustrates a method of providing direct-interface liquid cooling of an RIHS. In one or more embodiments, the method1500includes selecting one of a facility supply and return and a closed loop pumped configuration to move cooling liquid through a liquid rail of an RIHS (block1502). The method1500includes sensing a temperature of one or more of: a heat-generating functional component in an LC node, an air temperature within a rack in which the LC node is mounted, an ambient temperature outside of the RIHS, and a temperature of the cooling liquid (block1504). The method1500includes filtering the supplied cooling liquid using first and second filtration subunits of the RFU (block1506). The method1500includes detecting when one of the filtration units is removed from the liquid supply (block1508), and in response to that detection, diverting all of the cooling liquid to filter through the first filtration subunit to enable continued filtration, while the second filtration unit is removed and/or replaced (block1510). The method1500includes dynamically controlling a node-level supply valve to divert a portion of the cooling liquid flowing through a supply conduit of the liquid rail through an embedded heat exchange manifold attached in proximity to the heating-generating functional components in response to the sensed temperature (block1512). The method1500includes routing the returned cooling liquid through an air-liquid heat exchanger of a block liquid manifold to cool one of the forced cooling air and the cooling liquid before returning the cooling liquid to a return conduit of the liquid rail (block1514). Then method1500ends.

In the above described flow charts ofFIGS. 14-15, one or more of the methods may be embodied in an automated manufacturing system that performs a series of functional processes. In some implementations, certain steps of the methods are combined, performed simultaneously or in a different order, or perhaps omitted, without deviating from the scope of the disclosure. Thus, while the method blocks are described and illustrated in a particular sequence, use of a specific sequence of functional processes represented by the blocks is not meant to imply any limitations on the disclosure. Changes may be made with regards to the sequence of processes without departing from the scope of the present disclosure. Use of a particular sequence is therefore, not to be taken in a limiting sense, and the scope of the present disclosure is defined only by the appended claims.

One or more of the embodiments of the disclosure described can be implementable, at least in part, using a software-controlled programmable processing device, such as a microprocessor, digital signal processor or other processing device, data processing apparatus or system. Thus, it is appreciated that a computer program for configuring a programmable device, apparatus or system to implement the foregoing described methods is envisaged as an aspect of the present disclosure. The computer program may be embodied as source code or undergo compilation for implementation on a processing device, apparatus, or system. Suitably, the computer program is stored on a carrier device in machine or device readable form, for example in solid-state memory, magnetic memory such as disk or tape, optically or magneto-optically readable memory such as compact disk or digital versatile disk, flash memory, etc. The processing device, apparatus or system utilizes the program or a part thereof to configure the processing device, apparatus, or system for operation.